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FOREWORD 


The  Army  must  develop  leaders  who  can  effectively  apply  the  four  core  dimensions 
of  leadership:  values,  attributes,  skills,  and  actions.  These  provide  the  basis  for  leader 
development  policy,  doctrine,  training,  and  research.  To  assist  the  Deputy  Chief  of  Staff 
for  Personnel  (DCSPER)  as  the  proponent  for  leadership  and  leader  development  policy, 
the  US  Army  Research  Institute  (ARI)  and  George  Mason  University  (GMU)  sponsored  a 
workshop  of  military  and  civilian  leadership  researchers  on  November  12, 1998. 

This  report  summarizes  the  findings  of  the  Leadership  Tool  Kit  meeting.  This  was  a 
US  Army  Research  Institute  (ARI)  and  George  Mason  University  (GMU)  sponsored 
workshop  of  military  and  civilian  leadership  researchers.  This  group  identified  existing 
measures,  assessment  strategies,  or  measurement  templates  within  a  general  conceptual 
framework.  The  framework  was  organized  along  three  dimensions:  leadership  processes 
and  outcomes,  organizational  level,  and  level  of  analysis.  Several  measures  were 
recommended  and  reviewed  for  inclusion  base  on  criteria  such  as:  user  -  friendliness, 
breadth  of  scope,  face  validity,  existence  of  a  dociunented  research  record,  and  soundness 
of  psychometric  evidence.  The  resulting  battery  of  measures  was  then  analyzed  for  how 
well  they  fulfilled  all  elements  of  the  framework,  and  recommendations  were  made. 

The  report  provides  an  overview  of  1 5  leadership  assessment  measures  or  templates, 
with  descriptions,  summary  of  psychometric  evaluations,  application  and  source 
information,  and  references  and  suggested  reading  list.  While  measures  were  identified 
for  each  cell  of  the  conceptual  framework,  unit  leadership  at  the  upper  level  was  not  well 
assessed,  and  many  outcome  instruments  lacked  “hard”  behavioral  measures. 

Researchers  using  the  measures  in  this  tool  kit  are  asked  to  facilitate  their  further 
development.  This  research  was  briefed  to  the  Leadership  Action  Group,  chaired  by  BG 
Melton,  Director,  Human  Resources  Directorate,  DCSPER. 

It  is  hoped  that  this  leadership  assessment  tool  kit  will  help  all  organizations  to 
improve  current  leadership  research,  develop  new  ideas  about  leadership,  build 
consensus,  and  help  improve  the  development  of  effective  Army  leaders. 
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Developing  a  Tool  Kit  for  the  Assessment  of  Army  Leadership  Processes  and  Outcomes: 
Version  1 

EXECUTIVE  SUMMARY 


Research  Requirement: 

(a)  To  assist  the  Deputy  Chief  of  Staff  for  Personnel  (DCSPER)  as  the  proponent  for 
leadership  and  leader  development  policy  by  establishing  a  list  of  a  battery  of  measures 
that  have  been  identified  as  “best  practices”  for  assessing  leadership  effectiveness. 

Procedures: 

This  report  summarizes  the  result  of  a  US  Army  Research  Institute  (ARI)  and  George 
Mason  University  (GMU)  sponsored  workshop  of  military  and  civilian  leadership 
researchers.  This  group  identified  existing  measures,  assessment  strategies,  or 
measurement  templates  within  a  general  conceptual  framework.  The  framework  was 
organized  along  three  dimensions:  leadership  processes  and  outcomes,  organizational 
level,  and  level  of  analysis.  Several  measures  were  recommended  and  reviewed  for 
inclusion  base  on  criteria  such  as:  user  -  friendliness,  breadth  of  scope,  face  validity, 
existence  of  a  documented  research  record,  and  soundness  of  psychometric  evidence. 

The  resulting  battery  of  measures  was  then  analyzed  for  how  well  they  fulfilled  all 
elements  of  the  framework,  and  recommendations  were  made. 

Findings: 

The  report  provides  an  overview  of  15  leadership  assessment  measures  or  templates, 
with  descriptions,  summary  of  psychometric  evaluations,  application  and  source 
information,  and  references  and  suggested  reading  list.  While  measures  were  identified 
for  each  cell  of  the  conceptual  framework,  unit  leadership  at  the  upper  level  was  not  well 
assessed,  and  many  outcome  instruments  lacked  “hard”  behavioral  measures. 

Researchers  using  the  measures  in  this  tool  kit  are  asked  to  facilitate  their  further 
development.  It  is  hoped  that  the  structure  of  this  leadership  assessment  tool  kit  will 
assist  all  organizations  in  improving  current  leadership  research,  develop  new  ideas  about 
leadership,  build  consensus,  and  help  improve  the  development  of  effective  Army 
leaders. 

Utilization  of  F indings : 

This  research  was  briefed  to  the  Leadership  Action  Group,  chaired  by  BG  Melton, 
Director,  Human  Resources  Directorate,  DCSPER. 
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Developing  a  Tool  Kit  for  the  Assessment  of  Army  Leadership  Processes  and  Outcomes: 

Version  1.0 

Background 

The  U.S.  Army  has  had  a  long-standing  interest  in  leadership,  in  particular,  the 
assessment  of  leadership  constructs.  This  interest  has  resulted  in  a  proliferation  of  leadership 
measures  and  assessment  tools.  Recently,  Mathieu,  Klimoski,  Rouse,  and  Marsh  (1997) 
completed  a  review  of  leadership  assessment  research  sponsored  by  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  between  1987  and  1997.  The  goals  of  this 
review  were  to  (a)  identify  and  describe  major  themes  and  initiatives  by  ARI  leadership  labs;  (b) 
critically  analyze  specified  instruments;  (c)  compare  ARI  assessment  tools  against  external 
benchmarks;  and  (d)  propose  recommendations  for  future  leadership  research  by  ARI. 

Mathieu,  et  al,  (1997)  made  a  number  of  observations,  conclusions  and  recommendations 
from  their  review.  Some  of  their  major  points  were: 

•  Because  most  theoretical  bases  for  leader  assessment  research  conducted  at  ARI 
emphasized  the  change  in  leader  performance  requirements  across  careers  and 
organizational  hierarchies  (e.g..  Stratified  Systems  Theory),  measures  of  leader 
knowledge  and  characteristics  needed  to  be  developed  with  consideration  given  to 
contextual  variables; 

•  In  several  circumstances,  commercially  available  measures  of  leader  personality 
variables  are  preferable  to  measures  developed  internally  at  ARI; 

•  Several  measures  of  leader  knowledge  and  expertise  developed  through  ARI 
sponsorship  were  promising; 

•  A  multi-source  (i.e.,  360  degree)  rating  format  should  be  considered  for  the 
assessment  of  leadership  behavior; 

•  Working  with  a  research  protocol  had  advantages.  Specifically,  the  following 
research  protocol  was  recommended  for  research  directed  toward  the  development 
of  appropriate  standardizable  leadership  assessment  tools:  (1)  adopt  a  theory  or 
conceptual  framework  that  describes  the  antecedents  of  leader  effectiveness;  (2) 
Complete  an  updated  analysis  of  the  requirements  of  Army  leadership  positions  to 
identify  both  common  and  unique  dimensions;  (3)  Identify  and  analyze  the 
knowledge,  skills,  abilities,  and  other  attributes  important  for  accomplishing 
leader  performance  dimensions;  (4)  Develop  effective  criteria  measures  of 
effective  leadership;  (5)  Use  more  confirmatory  (and  less  exploratory)  methods 
for  evaluating  instrument  dimensionality;  and  (6)  Incorporate  additional 
explanatory  variables  to  identify  limits  to  generalizability  and  potential  moderator 
relationships; 

•  Give  greater  consideration  to  contextual  variables  (e.g.,  technology,  follower 


characteristics,  changes  in  operational  environments)  that  are  likely  to  change  in 
the  2 1  St  century,  particularly  to  the  influences  of  these  changes  on  Army 
leadership  and  leader  assessment; 

•  There  is  a  serious  need  for  the  development  of  leadership  measures  that  accurately 
appraise  the  effects  and  influences  of  leadership  on  performance. 

The  last  point  is  perhaps  the  most  critical  of  the  issues  raised  by  Mathieu  et  al.  (1997). 
Continuing  research  efforts  to  understand  Army  leadership  and  leader  development  need  to  be 
grounded  in  a  useable  set  of  criterion  measures  that  most  researchers  agree  have  conceptual 
fidelity  and  acceptable  psychometric  properties.  For  example,  without  valid  and  useful  criterion 
measures,  successful  research  cannot  be  conducted  to  develop  and  validate  the  meaningfulness  of 
assessments  of  leader  values,  knowledge,  skills,  personality  variables,  capabilities,  and  other 
personal  attributes.  Likewise,  on  a  more  practical  note,  leader  effectiveness  measures  are  also 
necessary  to  validate  conceptual  specifications  of  leader  selection  requirements. 

If  measures  of  leadership  effectiveness  are  indeed  inadequate,  then  research  efforts  need 
to  be  devoted  to  the  development  of  better  and  more  appropriate  assessment  tools  or  procedures. 
Such  efforts  could  yield  toolkits  that  can  serve  as  markers  of  "real  world"  effectiveness  (e.g., 
military  combat  effectiveness).  Accordingly,  the  initial  tasks  in  a  broad  and  comprehensive 
research  program  on  Army  leadership  are  to  (a)  identify  those  measures  that  can  be  considered 
"best  practices"  for  the  assessment  of  leadership  effectiveness;  and  (b)  develop  and  validate  new 
measures  of  leadership  effectiveness  that  are  not  adequately  represented  among  these  practices. 

Leader  Effectiveness  Workshop 

To  examine  current  and  potential  measures  of  leadership  effectiveness,  ARI  sponsored  a 
conference  of  military  and  civilian  leadership  researchers.  These  participants  were  asked  to 
organize  the  domain  of  leader  effectiveness  and  to  identify  suitable  existing  measures, 
assessment  strategies,  or  measurement  templates.  The  "criterion  space"  for  leader  effectiveness 
to  be  included  would  involve  tools  that  would  get  at  both  leadership  processes  (e.g.,  "planning", 
"communicating",  "motivating"),  and  outcomes  of  such  processes  (e.g.  "subordinate  motivation 
and  conunitment",  "unit  performance").  The  proceedings  of  this  workshop  are  summarized  in 
Appendix  A. 

The  intended  major  product  of  this  workshop  was  a  leader  assessment  tool  kit.  This  is  a 
set  of  measures  (or  tools)  or  measurement  approaches  that  represent  appropriate  ways  to  assess 
leader  effectiveness.  Such  a  tool  kit  could  be  expected  to  become  part  of  a  standardized  protocol 
to  be  used  in  leadership  research  endeavors  across  the  Army.  Standard  users  of  such  a  kit  would 
promote  the  development  of  a  more  comprehensive  data  and  knowledge  base  regarding  Army 
leadership  than  currently  exists.  A  standard  measurement  protocol  would  also  facilitate  the 
applicability  of  research  findings  across  Army  domains,  as  well  as  the  identification  of  factors 
that  restrict  such  applicability. 

The  present  report  summarizes  the  organization  and  the  contents  of  the  leadership 


2 


assessment  toolkit.  The  conceptual  framework  that  guided  the  organization  of  the  toolkit  is 
presented  in  the  next  section  of  this  report.  Then,  the  specific  measures  that  comprise  the  toolkit 
are  listed,  followed  by  some  general  conclusions. 

Conceptual  Framework  for  the  Proposed  Leadership  Assessment  Tool  Kit 

Figure  1  presents  the  conceptual  framework  proposed  for  the  organization  of  a  leadership 
assessment  took  kit  and  presented  to  conference  participants.  This  framework  is  organized  along 
three  dimensions:  leadership  processes  and  outcomes,  organizational  level,  and  level  of  analysis. 


Measures  of  Leader  Effectiveness 
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Figure  1 .  Conceptual  framework  for  organization  of  leadership  assessment  measures. 

Leadership  processes.  The  dimensions  of  leader  processes  versus  outcomes  emphasizes 
that  the  criterion  space  for  leader  effectiveness  can  be  divided  into  (a)  specific  actions  (e.g., 
interpersonal  processes,  personnel  management)  and  mental  processes  (e.g.,  decision  making, 
planning,  and  strategy  making)  that  constitute  the  practice  of  leadership,  i.e.,  leadership 
processes,  and  (b)  the  consequences  of  those  actions  and  mental  processes,  i.e.,  leadership 


outcomes.  In  thinking  about  leadership  in  terms  of  an  input-process-outcome  (IPO)  framework, 
leadership  process  measures  might  then  serve  as  criteria  for  the  influence  of  leader  attributes  and 
the  leader's  interpretations  of  role  performance  requirements.  In  many  conceptual  models  of 
leadership,  such  processes  are  considered  mediators  of  the  influence  of  leader  attributes  and 
performance  requirements  on  performance  outcomes  (see  Bass,  1990,  Yukl,  1994,  and  Zaccaro, 
1996,  for  reviews  of  such  models).  Accordingly,  measures  of  leadership  processes  are  important 
for  validating  instruments  developed  to  assess  leader  attributes. 

Leadership  outcomes.  Leadership  outcomes  are  defined  as  the  consequences  of 
leadership  actions  and  mental  processes.  Such  outcomes  can  be  specified  both  in  terms  of 
changes  in  psychological  states  as  a  result  of  leadership  influences  (e.g.,  changes  in  the  leader's 
or  subordinates'  cognition,  knowledge,  attitudes,  and  motivation;  also  developmental  growth), 
and  tangible  performance  (i.e.,  action-oriented)  results  (e.g.,  quantity,  quality,  mission 
accomplishment).  Conceivably,  some  outcomes  such  as  changes  in  leader  and  subordinate 
knowledge,  skills,  attitudes  and  motivation  can  be  construed  as  antecedents  of  other  outcomes 
(e.g.,  imit  mission  accomplishment).  However,  for  the  sake  of  parsimony,  this  level  of 
conceptual  specification  was  not  adopted  for  the  proposed  tool  kit.  It  should  be  acknowledged  at 
the  outset  that  while  outcomes  may  be  closely  coupled  to  leader  behaviors,  in  complex 
organizations  they  may  be  the  consequence  of  processes  or  choices  by  the  leader  made  much 
earlier  or  even  at  different  places  (e.g.,  a  decision  to  organize  a  unit  a  particular  way). 

Organizational  level.  The  nature  of  effective  leader  processes  and  outcomes  were 
considered  to  vary  by  organizational  level.  For  purposes  of  discussion,  we  specified  three  levels, 
lower,  middle,  and  upper  organizational  leadership.  This  stratification  corresponds  to  most 
models  of  organizational  leadership  (e.g..  Hunt,  1991;  Jacobs  &  Jaques,  1987,  1990, 1991; 
Zaccaro,  1996).  The  working  assumption  is  that  the  leadership  processes  necessary  for  the 
successful  accomplishment  of  work  requirements  vary  across  organizational  levels.  LikeAvise, 
the  outcomes  reflective  of  effective  junior  leadership  are  different  respectively  from  those  of 
effective  middle  or  senior  leadership.  Accordingly,  assessments  tools  used  to  measure  leadership 
processes  and  outcomes  need  to  vary  in  content  depending  upon  the  targeted  level  of 
organizational  leadership  of  interest. 

Level  of  analysis/Target  of  influence.  The  final  dimension  in  the  conceptual  fi-amework 
proposed  for  the  leader  assessment  tool  kit  organized  potential  measures  according  to  different 
levels  of  analysis  in  terms  of  targeted  influence.  Four  such  levels  were  specified:  individual, 
dyad,  team,  and  unit.  Individual-level  measures  would  include  those  instruments  that  assess 
intraindividual  processes  (e.g.,  cognitive  activities  of  environmental  scanning,  information 
organization,  as  well  as  behaviors  in  support  of  boundary  spanning  or  strategic  plaiming)  and 
outcomes  (e.g.,  the  leader's  developmental  and  professional  growth,  the  quality  of  his  or  her 
strategic  plans).  Dyadic-level  measures  focus  on  leader  subordinate  exchange  processes  (e.g., 
conummication,  developing  subordinate  skills)  and  the  products  of  such  processes  (e.g., 
subordinate  motivation,  subordinate  performance  quality).  Team-level  instruments  reflect  team 
leadership  processes  (e.g.,  norm  formation  and  enforcement,  team  decision-making)  and  team 
outcomes  (e.g.,  team  cohesion,  team  mission  accomplishment).  Unit-level  measures  target  the 
leader's  influence  over  large  scale  organizational  units.  Such  measures  may  focus  on  unit-level 
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processes  (e.g.,  development  of  organizational  strategy  or  vision,  distribution  of  information  and 
plans,  changes  in  organizational  structure  and  policy),  and  imit  outcomes  (e.g.,  work  flow, 
readiness  status,  organizational  mission  accomplishment). 

Summary  of  Tool  Kit  Contents 

The  authors  of  this  report  reviewed  the  recommendations  of  the  small  groups  at  the 
toolkit  workshop  and  the  evaluation  report  produced  by  Mathieu  et  al.  (1997).  From  these  and 
other  sources  of  information  about  leadership  and  leader  assessment  (e.g.,  Bass,  1990, 1996; 
Yukl,  1994,  Zaccaro,  1996),  they  compiled  a  set  of  proposed  measures  for  inclusion  in  a  leader 
assessment  toolkit.  These  measures  are  listed  in  this  section  of  the  report.  They  are  organized  by 
the  three  dimensions  of  the  conceptual  framework  (i.e.,  process  versus  outcome,  organizational 
level,  and  level  of  analysis  or  targeted  influence).  Appendix  B  contains  a  description  of  each 
measure,  summaries  regarding  its  psychometric  evaluations,  comments  regarding  application  of 
the  measure,  sources  to  obtain  the  measure,  and  references  and  suggested  readings  for  the 
measure. 

In  brief,  however,  the  following  measures  and  tools  are  being  offered  as  entries: 
Leadership  Processes 

Lower  level  leaders:  Individual  level  of  analysis 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Leader  Azimuth  Check 

•  Observer/Controller  ratings  -  template  required 

Lower  level  leaders:  Dyad  level  of  analysis 

•  LMX-7 

•  Managerial  Practices  Survey 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 

•  subordinate  ratings  from  360  degree  assessment  protocols  (e.g..  Leader 
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Azimuth  Check) 

Lower  level  leaders:  Team  level  of  analysis 

•  Team-LMX 

•  Managerial  Practices  Survey 
Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
Lower-level  leadership:  Unit  level  of  analysis 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
Leadership  Processes 

Middle  level  leaders:  Individual  level  of  analysis 

•  Multifactor  Lfeadership  Questionnaire 

•  Competing  Values  Inventory 

•  Leader  Azimuth  Check 

•  Observer/Controller  ratings  -  template  required 
Middle  level  leaders:  Dyad  level  of  analysis 

•  Managerial  Practices  Survey 

•  Multifactor  Leadership  Questionnaire 
LMX-7 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
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•  subordinate  ratings  from  360  degree  assessment  protocols  (e.g.,  Leader 
Azimuth  Check) 

Middle  level  leaders:  Team  level  of  analysis 

•  Team-LMX 

•  Managerial  Practices  Survey 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
Middle  level  leaders:  Unit  level  of  analysis 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
Leadership  Processes 

Upper  level  leaders:  Individual  level  of  analysis 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
Upper  level  leaders:  Dyad  level  of  analysis 

•  Multifactor  Leadership  Questionnaire 
LMX-7 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
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•  subordinate  ratings  from  360  degree  assessment  protocols  (e.g.,  Leader 
Azimuth  Check) 

Upper  level  leaders:  Team  level  of  analysis 

•  Team-LMX 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
Upper  level  leaders:  Unit  level  of  analysis 

•  Multifactor  Leadership  Questionnaire 

•  Competing  Values  Inventory 

•  Observer/Controller  ratings  -  template  required 
Leadership  Outcomes 

Lower  level  leaders:  Individual  level  of  analysis 

•  Benchmarks 

•  Observer/Controller  ratings  -  template  required 

•  TKML 

Lower  level  leaders:  Dyad  level  of  analysis 

•  LMX-7 

•  Empowering  Behavior  Questionnaire 

•  Observer/Controller  ratings  -  template  required 

•  Three-Component  Model  of  Organizational  Commitment 

•  Job  Descriptive  Index 

Lower  level  leaders:  Team  level  of  analysis 
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Team  Effectiveness  Profile 


•  Observer/Controller  ratings 

•  WRAIR  instrument 

•  mission  accomplishment  indices  -  template  required 

•  readiness  indices  -  template  required 
Lo^er  level  leaders:  Unit  level  of  analysis 

•  Observer/Controller  ratings 

•  WRAIR  instrument 

•  mission  accomplishment  indices  -  template  required 

•  readiness  indices  -  template  required 
Leadership  Outcomes 

Middle  Level  leaders:  Individual  level  of  analysis 

•  Observer/Controller  ratings  -  template  required 

•  Benchmarks 

Middle  Level  leaders:  Dyad  level  of  analysis 

•  LMX-7 

•  Empowering  Behavior  Questionnaire 

•  Observer/Controller  ratings  -  template  required 

•  Three-Component  Model  of  Organizational  Commitment 

•  Job  Descriptive  Index 
Middle  Level  leaders:  Team  level  of  analysis 

•  Team  Effectiveness  Profile 
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•  Observer/Controller  ratings  -  template  required 

•  WRAIR  instrument 

•  mission  accomplishment  indices  -  template  required 

•  readiness  indices  -  template  required 
Middle  Level  leaders:  Unit  level  of  analysis 

•  Observer/Controller  ratings  -  template  required 

•  WRAIR  instrument 

•  mission  accomplishment  indices  -  template  required 

•  readiness  indices  -  template  required 
Leadership  Outcomes 

Upper  Level  leaders:  Individual  level  of  analysis 

•  Observer/Controller  ratings  -  template  required 

•  Benchmarks  , 

Upper  Level  leaders:  Dyad  level  of  analysis 

•  LMX-7 

•  Empowering  Behavior  Questionnaire 

•  Observer/Controller  ratings  -  template  required 

•  Three-Component  Model  of  Organizational  Commitment 

•  Job  Descriptive  Index 
Upper  Level  leaders:  Team  level  of  analysis 

•  Team  Effectiveness  Profile 

•  Observer/Controller  ratings 

•  WRAIR  instrument 
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•  mission  accomplishment  indices  -  template  required 
readiness  indices  -  template  required 

•  ARI  command  climate  index 

Upper  Level  leaders:  Unit  level  of  analysis 

•  Observer/Controller  ratings  -  template  required 

•  WRAIR  instrument 

•  mission  accomplishment  indices  -  template  required 

•  readiness  indices  -  template  required 

•  ARI  command  climate  index 

General  Conclusions 

This  report  presents  a  proposed  toolkit  for  the  assessment  of  leader  effectiveness  in  the 
Army.  There  are  several  observations  to  be  made  about  this  toolkit.  First,  the  toolkit  is 
organized  around  three  dimensions,  or  questions  that  presumably  determine  whether  a  particular 
measure  is  appropriate  for  a  particular  research  application.  These  questions  are 

•  Are  the  leadership  effectiveness  criteria  of  interest,  process  variables,  outcome 

variables,  or  both? 

•  What  level  of  organizational  leadership  is  being  investigated? 

•  What  is  the  locus  or  target  of  leader  influence? 

Other  questions  would  also  dictate  the  utility  of  any  particular  measure.  However,  we  believe 
that  the  above  questions  are  the  most  critical  in  the  design  of  an  assessment  battery  in  any 
leadership  research  study.  Also,  these  questions  are  the  most  general  in  that  they  would  apply 
across  most  leadership  research  settings. 

The  cells  in  the  matrix  corresponding  to  measures  of  leadership  processes  displayed  by 
lower,  middle,  and  upper  level  leaders,  targeting  individual,  dyad,  and  team  levels  of  influence 
are  the  most  robust  in  terms  of  representation.  Measures  such  as  the  MLQ  and  the  MPS  have 
extensive  research  histories  with  well-established  psychometric  credentials.  Further,  while 
observer/controller  ratings  are  necessarily  context-specific,  templates  of  such  ratings  for  many 
military  settings  already  exist  and  can  be  adapted  readily  by  future  leadership  researchers.  Two 
issues  that  remain  about  the  measures  proposed  in  these  cells  is  that  (a)  as  a  set,  they  require 
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additional  research  to  verify  their  content  validity,  and  (b)  some  of  the  measures  (e.g.,  Leader 
Azimuth  Check)  still  require  additional  psychometric  investigations  before  their.widespread  use 
can  be  fully  endorsed. 

At  this  point,  it  should  be  noted  that  some  tools  and  instruments  do  appear  to  have 
general  applicability.  For  example,  the  MLQ  is  suitable  for  individuals,  dyads,  and  teams,  as 
well  as  for  many  levels  of  leadership  in  the  Army  --  lower,  middle,  and  upper.  Such 
generalization  can  promote  an  examination  of  key  research  findings  across  different  Army 
contexts.  However,  because  models  of  leader  effectiveness  are  likely  to  change  qualitatively 
according  to  the  dimensions  of  the  toolkit  matrix,  we  would  urge  more  research  on  assessment 
tools  and  instruments  that  are  specific  to  particular  organizational  levels  and  targets  of  leader 
influence. 

Also  noteworthy  is  that  we  discovered  that  when  the  level  of  analysis  is  unit  leadership, 
particularly  when  senior  leadership  processes  are  the  focus  of  research,  there  are  few  suitable 
candidates.  In  fact,  the  ones  selected  for  this  toolkit  are  the  same  as  those  for  leaders  at  other 
levels  of  analysis  and  of  the  organization.  But  these  measures  include  only  some  of  the 
processes  known  to  be  engaged  in  by  senior  leaders,  and  they  miss  those  processes  that  are 
unique  to  Army  executives.  The  use  of  observer/controller  ratings  is  primarily  speculative, 
because  few  examples  of  such  ratings  at  the  large  unit  level  exist  in  military  executive  leadership 
research.  Thus,  future  research  efforts  need  to  be  devoted  to  the  development  of  measures  that 
assess  leadership  processes  at  the  unit  level,  particularly  by  senior  leaders. 

The  toolkit  contains  a  conspicuous  lack  of  "hard”  or  behavioral  outcome  measures.  Most 
of  the  suggested  assessment  tools  reflect  attitudes  (e.g.,  JDI),  motivation  (e.g.,  Three-Component 
Model  of  Organizational  Commitment),  or  changes  in  attributes  as  a  result  of  leadership  training 
or  work  experience  (e.g..  Benchmarks).  The  OC  ratings  and  indices  reflecting  mission 
accomplishment  and  unit  readiness  are  the  only  suggested  performance  outcome  measures 
suggested  in  the  toolkit,  and  these  are  offered  only  in  the  form  of  assessment  templates.  Further, 
the  proposed  outcome  measures  do  not  differ  by  organizational  level  despite  considerable 
research  showing  that  the  nature  of  leadership  and  leader  performance  requirements  changes 
qualitatively  across  the  organizational  hierarchy.  Thus,  this  part  of  the  toolkit  requires  a 
substantial  investment  in  research  to  identify  and  validate  appropriate  level-specific  measures  of 
"objective"  or  "hard"  leader  performance  outcomes. 

The  notion  of  a  "template"  being  offered  in  various  cells  of  the  matrix,  in  lieu  of  any 
actual  measures,  deserves  some  comment.  In  particular,  we  have  suggested  that  observer/ 
controller  ratings,  mission  accomplishment,  and  readiness  indices  be  used  as  measures  of  leader 
effectiveness.  Because  these  measures  are  so  context-specific,  is  impossible  to  provide  a  single 
tool  that  is  applicable  across  all  leader  performance  domains.  However,  several  basic  dimensions 
can  be  identified  that  are  general  characteristics  of  effective  leader  performance.  The 
measurement  of  these  characteristics  should  be  the  core  of  an  assessment  template.  Two 
examples,  shown  in  Appendix  C,  are  the  observer/controller  rating  forms  developed  by  Bass, 
Avolio,  and  their  colleagues  to  assess  (a)  platoon  leader  performance,  and  (b)  performance  at  the 
National  Training  Center  (NTC),  respectively.  Each  form  assesses  basic  leadership  cognitive 
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and  interpersonal  processes.  The  platoon  performance  form  also  assesses  overall  platoon 
effectiveness.  However,  each  form  contained  exemplars  of  leadership  that  were^specific  to  the 
training  contexts  being  examined.  The  forms  also  differed  in  that  overall  effectiveness  was  not 
assessed  on  the  NTC  form,  while  the  platoon  performance  form  did  not  contain  Army  leadership 
doctrine.  We  urge  that  research  be  conducted  to  identify  critical  features  that  should  be  included 
such  OC  rating  forms  ~  the  use  of  such  forms  is  widespread  within  the  Army  and  they  can  be 
effective  measures  of  leadership  processes  and  outcomes.  However,  research  should  be  directed 
at  their  standardization  (as  much  as  is  possible,  given  different  training  context)  and  validation. 
Further,  similar  templates  need  to  be  developed  for  indices  of  mission  accomplishment  and  unit 
readiness. 

Next  Steps 

What  are  some  potential  next  steps  in  the  development  and  use  of  this  toolkit?  First,  the 
measures  offered  in  this  toolkit  will  initially  need  to  be  used  with  caution.  Researchers  who 
begin  to  use  particular  measures  on  a  regular  basis  should  establish  local  norms  and  data  for 
subsequent  interpretation  and  application.  These  data  should  also  contribute  evidence  for  the 
continued  psychometric  viability  of  the  measure.  To  facilitate  the  continued  development  of 
toolkit  measures,  investigators  are  urged  to  provide  feedback  to  ARI  on  their  use  of  measures.  A 
feedback  form  located  at  Appendix  D  is  presented  for  this  purpose.  Second,  the  toolkit  should  be 
reviewed  periodically,  and  updated  to  reflect  the  latest  psychometric  findings  and  research 
activities. 

Third,  we  urge  that  researchers  who  use  this  measures  work  together  to  establish  a 
centralized  database  of  leadership  measures.  Such  a  database  can  contribute  to  integrated  studies 
across  different  Army  leadership  research  units,  as  well  as  more  longitudinal  and  career-long 
studies.  For  example,  common  measures  (e.g.,  the  MLQ,  MPS,  and  OC  ratings)  that  are 
administered  to  West  Point  and  ROTC  cadets,  officers  participating  in  the  Officer  Basic  and 
Advanced  course,  and  at  other  similar  points  in  an  officer's  career  can  provide  a  powerful  data  set 
that  lead  to  a  better  understanding  of  issues  related  to  leader  performance  and  leader 
development.  As  an  organization,  the  U.S.  Army  is  perhaps  uniquely  positioned  and  qualified  to 
provide  such  a  database.  This  and  similar  kinds  of  databases  can  greatly  enhance  our 
understanding  of  organizational  leadership. 

A  fourth  potential  next  step  is  to  develop  better  measures  to  fill  in  various  holes  in  the 
toolkit  matrix.  Related  to  this  recommendation  is  the  development  of  templates  for  OC  ratings, 
mission  accomplishment  measures,  and  readiness  indices.  As  we  noted,  the  matrix  does  not  offer 
many  assessment  tools  that  are  specific  to  particular  levels  of  Army  leadership.  However,  a  large 
body  of  research  has  established  that  different  leadership  processes  and  outcomes  characterize 
upper,  middle,  and  lower  level  leaders  (Zaccaro,  1996).  This  degree  of  differentiation  should  be 
reflected  more  accurately  in  the  toolkit. 

As  we  pointed  out  in  the  introduction,  one  desirable  assessment  feature  is  the  ability  to 
track  growth  and  learning  outcomes,  especially  those  related  to  self-development  and  distance 
learning  efforts.  As  the  Army  is  transforming  itself  into  a  learning  organization  with  increased 
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digitization  and  use  of  the  internet,  leader  development  activities  and  experiences  need  to  be 
integrated  into  the  leader  assessment  and  self-assessment  process.  One  key  to  making  this 
connection  is  the  assessment  of  leaders'  knowledge.  As  Mathieu  et  al.  (1997)  indicate,  leader 
knowledge  assessment,  using  tacit  knowledge  instruments  such  as  the  TBCML  show  considerable 
promise.  These  assessments  require  a  substantial  development  effort  because  of  their  specificity, 
and  the  subtle  nuances  of  the  test  require  validation  against  the  consensus  of  experts  or  large 
pools  of  leaders.  Although  this  has  been  done  successfully  for  lower  and  middle  leaders,  it 
remains  to  be  carried  out  for  upper  level  leaders,  particularly  on  strategic  leadership  issues. 

Finally,  the  report  by  Mathieu  et  al.  (1997)  that  evaluated  leader  assessment  research 
sponsored  by  ARI  argued  that  commercially-available  measures  may  be  preferable  to  those 
developed  internally  at  ARI.  Similarly,  we  would  suggest  a  moratorimn  on  the  development  of 
additional  "home  grown"  measures.  While  these  measures  have  the  advantage  of  being  more 
context-sensitive,  they  require  much  more  psychometric  investigation  that  most  researchers  can 
afford  to  complete  in  the  typical  time-pressured  environment  characterizing  most  Army 
leadership  research  endeavors.  Most  commercially  available  measures  have  already  developed 
substantial  psychometric  evidence  for  their  reliability  and  validity.  Further,  these  measures  can 
often  be  readily  adapted  to  military  situations  without  appreciable  loss  of  quality.  One  example 
of  such  application  is  the  MLQ,  offered  as  a  candidate  tool  in  all  of  the  leadership  process  cells 
in  the  toolkit  matrix.  While  this  measure  has  been  used  in  many  civilian  organizations,  it  has 
also  been  successfully  adapted  for  use  in  Army  platoons,  companies  and  battalions.  Its 
psychometric  properties  have  been  investigated  extensively. 

Summary 

This  report  offers  a  toolkit  of  leadership  measures  that  we  believe  is  suitable  for  research 
on  leadership  in  the  Army.  While  there  remains  a  substantial  amount  of  work  that  needs  to  be 
completed  to  expand  and  validate  parts  of  this  toolkit  (i.e.,  to  add  more  and  better  tools),  the 
measures  included  here  can  provide  the  basis  for  a  powerful  research  database  on  Army 
Leadership.  Such  information  source  can  be  invaluable  to  Army  policy  makers  as  they  make 
decisions  about  leadership  and  leadership  development  in  the  21st  century. 
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APPENDIX  A 


Proceedings  of  the  Leadership  Assessment  Toolkit  Workshop 

The  toolkit  presented  in  this  emerged  in  part  from  a  workshop  sponsored  by  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  and  George  Mason  University. 
Participants  in  this  workshop  included:  active  duty  Army  officers,  leadership  researchers  from 
multiple  Army  domains  (e.g.,  Army  Research  Institute,  Center  for  Army  Leadership,  U.S. 
Military  Academy),  leadership  researchers  from  multiple  academic  settings,  leadership 
consultants  and  practitioners,  and  Army  persoimel  responsible  for  the  education  and 
development  of  junior  army  officers.  A  listing  of  all  participants  is  given  in  Appendix  E.  This 
appendix  to  the  report  describes  the  proceedings  of  this  workshop. 

Pre-workshop  Activities 

Prior  to  arriving  at  the  workshop,  participants  were  asked  to  review  materials  outlining 
the  nature  of  the  tasks  to  be  accomplished  when  they  arrived  and  an  organizing  scheme  of  the 
expected  work  product  from  their  efforts.  They  were  also  asked  to  do  some  preparation  for  the 
session  (see  Appendix  F).  Specifically,  with  regard  to  the  latter,  they  were  asked  to  nominate 
candidate  measures  for  inclusion  in  the  toolkit.  They  were  asked  to  categorize  their  nominations, 
according  to  their  appropriateness  or  applicability  to  certain  levels  of  organizational  leadership, 
targets  of  leader  influence,  and  whether  the  candidate  measure  assessed  leadership  processes, 
outcomes,  or  both.  In  making  nominations,  participants  were  advised  that: 

•  a  given  measure  could  be  nominated  for  more  than  one  cell; 

•  more  than  one  measure  could  be  nominated  for  each  cell; 

•  some  cells  were  likely  to  remain  empty  despite  the  efforts  of  participants; 

•  candidate  measures  may  be  derived  from  basic  or  applied  research; 

•  candidate  measures  may  come  from  either  (or  both)  military  and  civilian 
leadership  research. 

Participants  were  asked  to  cite  relevant  sources  and  references  for  each  candidate 
measure.  They  were  also  asked  to  bring  to  the  conference  a  specimen  copy  of  each  proposed 
measure.  Appendix  F  contains  the  background  information  and  assignment  instructions  mailed 
to  each  of  the  participants. 

The  three  dimensions  of  organizational  level,  target  of  influence,  and  processes  versus 
outcomes  were  offered  to  workshop  participants  as  the  primary  means  of  thinking  about  and 
classifying  potential  candidates  for  the  proposed  leader  assessment  toolkit.  But,  other 
considerations  or  factors  were  also  entertained  for  such  a  classification.  These  include  (a)  the 
presumed  puipose  or  "use"  of  the  measure  (e.g.,  to  measure  effectiveness  in  training  or  for 
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administrative  actions,  like  promotion  decisions);  (b)  the  setting  or  venue  for  the  use  of  a 
proposed  measure  (e.g.,  Army  garrison  context,  war  games/simulation;  (c)  targeted  sample  (e.g., 
military  versus  Army  civilian  leaders);  and  source  of  effectiveness  data  (e.g.,  peers,  supervisors, 
subordinates,  self,  observers,  archival  records).  While  important,  these  factors  were  treated  as 
secondary  to  the  three  major  aforementioned  dimensions.  Nonetheless,  workshop  participants 
were  also  asked  to  consider  these  factors  in  their  deliberations  of  various  measures  and  of  the 
utility  of  the  proposed  toolkit. 

Workshop  Proceedings 

Workshop  participants  were  asked  to  complete  the  following  tasks  at  the  event  itself: 

•  Describe  current  Army  practices  in  the  assessment  of  leadership  processes  and 
outcomes; 

•  Consider  the  implications  and  usefulness  of  a  heuristic  framework  for  organizing 
existing  tools,  techniques,  and  measures  for  assessing  Army  leader  effectiveness; 

•  Using  this  framework,  identify  potential  "best  practices"  in  assessing  leadership 
performance;  aind 

•  Contribute  to  a  "draft"  tool  kit,  organized  around  the  framework.  The  content  of 
the  tool  kit  will  reflect  potential  appropriate  Army  leader  assessment  tools,  and 
indicate  shortfalls  signaling  future  research  needs. 

In  addition  to  the  development  of  a  potential  leadership  effectiveness  assessment  tool  kit, 
the  workshop  had  several  other  ancillary  objectives.  One  was  to  provide  a  guide  for  active  Army 
researchers  to  shape  their  studies  on  leadership.  More  specifically,  this  objective  was  intended  to 
promote  a  greater  alignment  of  research  activities  and  goals  to  foster  models  that  view  the 
training,  development  and  manifestation  of  leadership  in  the  Army  more  systematically.  Thus, 
the  research  questions  being  addressed  at  various  labs  and  field  sites  could  be  coordinated  in 
order  to  address  Army  needs.  The  workshop  was  also  intended  to  be  a  forum  for  comparing 
different  research  programs  among  active  leadership  researchers  in  both  the  Army  and  academic 
communities,  to  uncover  research  priorities  particularly  in  the  field  of  leadership  assessment,  and 
to  stimulate  future  research  collaborations  among  participants  in  these  two  communities. 

The  agenda  of  the  workshop,  shown  in  Appendix  G,  was  developed  to  promote  a 
stimulating  and  expert  exchange  about  leadership  assessment,  particularly  in  the  Army.  During 
the  workshop,  the  initial  sessions  were  plenary  ones  organized  around  three  themes:  (a)  current 
leader  assessment  practices  in  the  Army,  (b)  criteria  for  effective  assessment  instruments,  and  (c) 
nominations  of  measures  for  potential  inclusion  in  the  proposed  toolkit.  The  plenary  sessions 
were  followed  by  small  group  discussions  in  which  participants  considered  the  strength  and 
weaknesses  of  particular  measures  and  made  recommendations  regarding  their  suitability 
according  to  criteria  established  during  the  plenary  sessions.  At  the  conclusion  of  the  workshop, 
each  group  summarized  their  recommendations. 
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Workshop  plenary  session:  Current  practices  and  criteria  for  candidate  measures. 

After  introductions,  the  first  major  session  focused  on  a  discussion  of  current  practices  in  the 
Army  pertaining  to  leader  assessment.  This  discussion,  led  primarily  by  active  duty  and  retired 
army  officers,  ended  up  being  focused  on  the  following  topics:  (a)  Officer  Effectiveness  Ratings 
(OER);  (b)  informal  assessments  by  commanding  officers  for  development  purposes;  (c) 
subordinate  ratings  of  commanding  officers;  (d)  observer/controller  (OC)  rating  procedures;  and 
(e)  after  action  reviews  (AAR).  Each  of  these  procedures  were  described,  including  a  summary 
of  their  perceived  utility  in  assessment  as  well  as  their  disadvantages.  Some  conclusions  from 
this  discussion  were: 

•  leader  assessment,  whether  formal  or  informal,  is  an  ongoing  activity  in  the 
Army; 

•  most,  if  not  all,  assessment  procedures  in  active  use  emphasized  primarily 
leadership  processes,  not  outcomes; 

•  OERs,  as  the  primary  leader  assessment  tool  in  the  Army,  were  judged  as  a 
flawed  system  because  of  the  dichotomous  choices,  lack  of  variability  in  ratings, 
lack  of  a  benchmark  standard,  and  its  lack  of  differentiation  across  organizational 
levels.  The  "new"  OER  procedures,  including  a  greater  emphasis  on  discipline  in 
the  ratings,  were  judged  as  an  improvement; 

•  often,  formal  assessments  (i.e.,  OERs,  OC  ratings,  AARs)  were  used  as  a  context 
for  leader  development  action  plans;  these  formal  assessments  were  sometimes 
augmented  by  informal  assessment  procedures  designed  by  the  administrating 
officer;  and 

•  OC  ratings  were  viewed  as  potentially  useful  means  for  leader  process  and 
outcome  assessments  within  training  contexts.  Discussants  promoted  the  utility 
of  a  generic  "template"  for  OC  ratings  that  can  provide  information  across 
different  performanee  eontexts. 

The  discussion  generally,  however,  reflected  a  consensus  that  current  assessment 
practices  were  insufficient  to  meet  the  needs  of  the  Army  of  the  future,  and  that  new  measures, 
particularly  of  leadership  outeomes,  were  neeessary  for  more  effective  evaluations  of  leadership 
in  training  and  operational  settings. 

The  discussion  was  then  direeted  toward  the  articulation  of  standards  or  criteria  for 
effeetive  leader  assessment  tools:  Partieipants  suggested  that: 

•  assessments  should  be  fairly  parsimonious  and  user-friendly; 

•  assessments  should  be  broad  in  scope,  content  valid,  and  be  adaptable  for  military 
settings  (i.e,.  they  should  have  military  face  validity); 
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•  to  be  defensible,  tools  should  be  built  upon  a  documented  research  record  with 
sufficient  psychometric  evidence; 

•  assessments  should  not  be  susceptible  to  range  restriction  and  other  measurement 
biases; 

•  desirable  assessment  procedures  include  those  that  allow  users  to  track  growth 
and  learning  outcomes,  including  those  related  to  self-development  and  distance 
learning  efforts;  and 

•  absent  a  specific  tool  that  would  work  for  several  contexts  or  levels,  user-friendly 
templates  should  be  developed  for  those  assessment  procedures  that  are 
necessarily  context  specific. 

Workshop  plenary  session:  Nominations  of  candidate  measures.  After  discussions 
of  current  assessments  practices  in  the  Army  and  criteria  for  candidate  measures,  workshop 
participants  nominated  measures  for  inclusion  in  the  leader  assessment  toolkit.  These 
nominations  were  the  basis  for  small  group  discussions  during  the  remainder  of  the  workshop. 
Participants  were  initially  asked  to  make  nominations  without  detailed  consideration  of  a 
measure's  psychometric  quality,  or  of  the  criteria  listed  above.  These  considerations  were  to  be 
considered  later  as  part  of  the  tasks  assigned  to  the  small  groups.  Table  1  summarizes  the 
measures,  indicated  mostly  by  acronym,  nominated  in  each  cell  of  the  conceptual  framework. 
Table  2  lists  the  acronym  and  title  for  each  of  the  measures. 

From  Tables  1  and  2,  it  is  clear  that  participants  nominated  a  large  number  of  measures, 
with  a  minimum  of  seven  in  any  one  cell  in  the  conceptual  framework.  Both  leadership 
processes  and  outcomes  were  amply  represented,  as  were  measures  across  organizational  levels. 
It  should  be  noted  again  that  these  nominations  represented  the  starting  points  for  small  group 
discussions,  not  final  endorsements.  Indeed,  as  a  result  of  careful  scrutiny,  many  of  these 
measures  were  abandoned  in  final  recommendations. 

Workshop  small  group  discussions.  Four  groups  were  established  for  the  break-out 
discussion  period.  Each  group  was  assigned  one  level  of  analysis  (i.e.,  individual,  dyad,  team, 
and  unit)  in  the  conceptual  framework  for  their  deliberations.  They  were  asked  to  examine 
process  and  outcome  measures  for  lower,  middle,  and  upper  level  leaders  at  the  group's  assigned 
level  of  analysis.  The  small  groups  were  also  given  the  following  charges: 

•  Consider  the  nominations  in  particular  cells  of  the  proposed  toolkit  matrix, 

•  Add  additional  nominations  if  necessary, 

•  Consider  criteria  for  a  good  tool/measure  as  discussed  earlier, 

•  Come  up  with  appropriate  templates,  where  no  specific  measures  have  been 
nominated. 
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•  Classify  nominations  in  terms  of  preferences  and  recommendations  for  further 
consideration,  and 

•  Prepare  for  and  lead  a  discussion  of  the  group's  recommendations  at  a  later 
plenary  session. 

The  members  of  each  of  the  four  discussion  groups  are  indicated  in  Appendix  H. 

Each  of  the  groups  followed  a  similar  plan.  First,  reflecting  on  the  nature  of  effective 
leadership  at  the  organizational  level  of  interest,  group  members  articulated  and  discussed  the 
leadership  processes  and  outcomes  that  needed  to  be  considered  in  an  assessment  battery;  i.e., 
they  focused  on  the  content  of  proposed  measures.  Tables  3-6  indicated  both  the  processes  and 
outcomes  that  were  decided  as  the  appropriate  content  for  measures  at  the  individual,  dyad,  team, 
and  unit  level,  respectively. 

The  small  groups  then  evaluated  each  of  the  measures  nominated  for  their  assigned  level 
of  analysis  during  the  earlier  session  (see  Tables  1  and  2)  according  to  the  content  they  defined 
earlier.  They  identified  appropriate  candidates  for  the  proposed  leader  assessment  toolkit  and, 
perhaps  more  importantly,  defined  areas  where  measurement  development  research  was 
necessary.  Each  group  then  presented  its  findings  to  all  of  the  workshop  participants  in  a  final 
plenary  session.  The  toolkit  presented  in  this  report  was  developed  in  part  from  these  findings. 
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Measures  of  Leader  Effectiveness  (continued) 


Table  2 


Nominated  Assessment  Measures 


Acronym 

Title 

ACC 

Adaptive  Coping  Cycle 

ACCESS 

Command  and  Control  Evaluations  Systems  Decision  Cycles 

AZIMUTH 

Leader  Azimuth  Check  II 

Benchmarks 

Benchmarks 

CLI 

Campbell  Leadership  Index 

CM 

Conflict  Management 

Command  Climate 

U.  S.  Army  Automated  Command  Climate  Survey 

CUS 

Campbell  Unit  Survey 

CVI 

Competing  Values  Framework 

ECATS 

Climate  Survey 

Empower 

Empowering  Behavior  Questionnaire 

JDI 

Job  Descriptive  Index 

JSS 

Job  Stress  Survey 

LBDQ 

Leader  Behavior  Description  Questionnaire 

LMX-7 

Leader  Member  Exchange-7 

MEI 

Meeting  Effectiveness  Inventory 

MPS 

Managerial  Practices  Survey  or  Compass 

Mission 

Mission  Accomplishment 

MLQ 

Multifactor  Leadership  Questiormaire 

MLQT 

Multifactor  Leadership  Questionnaire  for  Teams 

OCB 

Organizational  Citizenship  Behavior 

OC  Ratings 

Observer  Controller  Ratings 

OCQ 

Organizational  Commitment  Questionnaire 

Readiness 

Readiness  Indices 

Resource 

Resource  Consumption 

S/H 

Shamir-Hunt  Charisma  Instrument 

SLDI 

Strategic  Leader  Development  Inventory 

TARGET 

Simulated  Combat  Measure 

Team  LMX-7 

Team  Leader  Member  Exchange-7  (adapting  LMX-7) 

TES 

Team  Effectiveness  Survey 

TKI 

Tacit  Knowledge  Inventory 

TLI 

Team  Leader  Inventory 

360° 

360  Degree  Assessment 
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Summary  Description  of  Selected  Leader  Assessment  Measures 


Assessment  Measures  Appendix 

Benchmarks  B1 

Competing  Values  Framework  B2 

Empowering  Behavior  Questionnaire  B3 

Job  Descriptive  Index  (JDI)  B4 

Leader  Azimuth  Check  II  (AZIMUTH)  B5 

Leader  Member  Exchange-7  (LMX-7)  B6 

Managerial  Practices  Survey  (MPS)  B7 

Multifactor  Leadership  Questioimaire  (MLQ)  B8 

Observer/Controller  Ratings  (Q/C  Ratings)  B9 

T acit  Knowledge  Measure  of  Leadership  (TKML)  BIO 

Team  Effectiveness  Survey  (TES)  B1 1 

Three-Component  Model  of  Organizational  Commitment  B12 

U.  S.  Army  Automated  Command  Climate  Survey  B13 

W alter  Reed  Army  Institute  of  Research  ( WRAIR)  B 1 4 

Leadership  Assessment  Tool 


360  Degree  Assessments 
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Reliability: 


Validity: 


Limitations: 


Application: 


Benchmarks 

Benchmarks  measures  a  wide  spectrum  of  management  behayiors.  The 
manager  and  the  manager’s  co-workers  (peers,  direct-report  boss) 
complete  the  questionnaire.  Benchmarks  were  constructed  by  studying 
managerial  deyelopment.  This  methodology  broadened  the  scope  of  the 
instrument  to  include  managerial  yalues  and  perspectiyes  as  well  as  skills. 
The  questionnaire’s  purpose  is  twofold:  to  help  identify  trouble  spots  in 
potentially  derailing  managers  and  to  suggest  ways  to  build  on  the 
manager’s  strengths.  The  respondent’s  score  is  compared  to  others’ 
ratings  and  to  the  Center  of  Creatiye  Leadership  norms.  Foreign 
benchmark  yersions  are  ayailable  for  English-speaking  managers  to  get 
feedback  from  international  raters. 

Studies  by  McCall  &  Lombardo  (1983)  and  McCall,  Lombardo,  & 
Morrison  (1988)  indicate  moderate  to  high  reliability  for  the  scales. 

-  Internal  consistency  -  Ayerage  alpha  =  .88 

-  Ayerage  test-retest  coefficient  for  self-ratings  =  .72 

-  Ayerage  test-retest  coefficient  for  ratings  by  others  =  .85 

-  Ayerage  Interrater  agreement  =  .58. 

Construct  yalidity  was  established  by  testing  hypothesis  about  group 
differences  on  scale  scores.  The  ayerage  ratings  on  Benchmarks  by  co¬ 
workers  for  1 1 1  managers  were  correlated  with  scores  on  three  scales:  the 
Meyers-Briggs  Type  Indicator  (MBTI);  the  Kirton  Adaption-Iimoyation 
Inyentory  (KAI),  and  the  Shipley  Institute  of  Liying  Scale.  Criterion 
yalidity  correlated  Benchmarks  with  criterion  measures  such  as:  bosses 
assessment  of  promotability,  independent  criterion  of  promotability, 
performance  eyaluation,  and  no  promotion.  The  highest  correlation  was 
with  bosses  assessment  of  promptability;  a  multiple  R  of  .46.  (McCauley 
&  Lombardo,  1990) 

Some  of  the  dimensions  appear  to  be  less  predictiye  of  managerial  success 
(e.g.,  balance  between  personal  life  and  work,  oyer  dependence,  and 
setting  deyelopmental  climate). 

Benchmarks  can  be  used  for  leader  feedback  and  deyelopment  purposes. 
The  assessment  tool  should  be  completed  by  the  leader  being  eyaluated 
and  by  his  or  her  peers. 
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Competing  Values  Framework 

The  Competing  Values  Framework,  deyeloped  by  Quinn  &  Rohrbaugh 
(1981, 1983),  purports  four  representatiye  yalue  dimensions  are  widely 
employed  in  organizations.  Each  quadrant  of  the  framework  represents 
one  of  four  major  models  of  organization  and  management  theory:  a 
human  relations  model,  an  open  systems  model,  an  internal  process  model, 
and  a  rational  goal  model.  The  human  relations  and  the  open  systems 
models  share  an  emphasis  on  flexibility,  while  the  open  systems  and  the 
rational  goal  models  haye  an  external  focus.  This  framework  is  labeled 
“Competing  Values”  because  the  criteria  seem  to  carry  a  seemingly 
conflicting  message  that  effectiye  performance  requires  both  flexibility 
and  stability  and  control. 

The  Competing  Values  Framework  also  seryes  as  a  useful  tool  to  organize 
executiye  leadership  (Hart  &  Quinn,  1993).  Like  organizational  styles,  the 
framework  suggests  that  there  should  be  a  balanced  yiew  of  executiye 
leadership  roles.  When  using  the  framework  to  integrate  the  leadership 
literature,  a  more  complete  picture  of  executiye  leadership  roles  emerges. 
Each  quadrant  of  the  model  represents  a  domain  of  action  with  a  role  for 
the  executiye.  Within  the  four  domains,  the  model  posits  that  there  are 
four  competing  demands  that  all  executiye  leaders  face:  innoyation, 
commitment,  efficiency,  and  performance.  The  roles  associated  with  these 
demands  are:  Vision  Setter,  Motiyator,  Analyzer,  and  Task  Master. 

The  Competing  Values  Framework  assumes  that  tensions  exist  among 
competing  needs,  tasks,  and  perceptions.  The  framework  can  assist  people 
in  graphing  their  perceptions  of  their  tensions.  Indiyiduals  who  engage  in 
diagnosis  bring  a  particular  conceptual  framework  to  the  task,  eyen  though 
they  may  not  be  conscious  of  their  attitudes.  This  diagnostic  framework 
using  a  competing  methodology  proyides  simple,  concrete, 
comprehensiye,  and  easily  applied  model  for  organizational  diagnosis  and 
analysis.  The  model  proyides  a  common  language  for  discussions  of 
organizational  effectiyeness  and  performance  as  well  as  for  facilitating 
communication  about  organizational  performance. 

Hart  &  Quinn  (1993)  deyeloped  a  16  item  questionnaire  measuring  the 
four  roles  of  leadership:  Vision  Setter,  Motiyator,  Analyzer,  and  Task 
Master.  Internal  consistency  yalues  for  Vision  Setter,  Motiyator, 

Analyzer,  and  Task  Master  were  .56,  .71,  .69,  and  .58,  respectiyely. 

Hart  &  Quinn  (1993)  demonstrated  construct  yalidity  of  the  four 
leadership  roles  performing  a  factor  analysis  on  the  items.  Four  factors 
emerged  clearly  reflecting  the  Vision  Setter,  Motiyator,  Analyzer,  and 
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Limitations: 


Application: 


Source: 


Task  Master  executive  roles.  The  authors  also  demonstrated  criterion- 
related  validity  of  three  of  the  four  executive  roles.  Vision  Setter  was 
fomid  to  predict  business  performance  (R^=.12,/?<.01)  and  organizational 
effectiveness  (R^=.  1 2,  p<.0 1 );  Motivator  was  found  to  predict 
organizational  effectiveness  (R^=.23,/><.001);  and  Analyzer  was  found  to 
predict  business  performance  (R^=.12,  j3<.01). 

The  role  of  Task  Master  did  not  demonstrate  criterion-related  validity 
(Hart  &  Quinn,  1993).  Further,  future  research  needs  to  be  conducted  on 
the  specific  relationships  that  exist  between  the  four  executive  roles  and 
organizational  performance.  For  example,  it  would  be  helpful  to  know  if 
different  executive  leadership  roles  were  required  at  different  stages  of  an 
organization’s  cycle  or  for  organizations  in  different  industries. 

The  Competing  Values  Framework  is  a  self-assessment  tool  that  can  be 
used  as  to  help  executives  assess  their  different  leadership  roles  in  an 
organization  and  evaluate  how  these  different  roles  contribute  to 
organizational  effectiveness.  The  information  provided  by  the  assessment 
can  be  used  for  developmental  purposes. 

Robert  E.  Quinn 

University  of  Michigan 

School  of  Business  Administration,  RM  3084 

914  Hill  Street 

Ann  Arbor,  MI  48109-1234 

Phone:  734-998-8159 

e-mail:  requinn@umich.edu 
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Empowering  Behayior  Questionnaire 

The  Empowering  Behayior  Questionnaire  is  a  multidimensional  measure 
of  psychological  empowerment  in  the  workplace.  Originally 
conceptualized  by  Thomas  and  Velthouse  (1990)  and  later  deyeloped  by 
Spreitzer  (1995),  psychological  empowerment  is  a  motiyational  construct 
inyolying  four  cognitions:  meaning,  competence,  self-determination,  and 
impact.  These  four  cognitions  reflect  an  actiye  role  of  work  orientation. 

The  meaning,  competence,  self-determination,  and  impact  scales  each 
consists  of  three  items  adopted  from  other  measures.  The  meaning  scale 
contains  three  items  deyeloped  by  Tymon  (1988);  the  competence  scale 
contains  three  items  from  Jones’s  (1986)  self-efficacy  scale;  the  self- 
determination  scale  contains  three  items  from  Hackman  and  Oldham’s 
(1985)  autonomy  scale;  and  the  impact  scale  contains  three  items  from 
Ashforth’s  (1989)  helplessness  scale. 

Coefficient  alpha  reliability  analysis  was  conducted  on  two  samples  of 
mid-leyel  employees  from  an  industrial  and  insurance  organization.  The 
oyerall  empowerment  construct  was  .72  for  the  industrial  sample  and  .62 
for  the  insurance  sample.  Test-retest  reliability  was  performed  for  the 
insurance  sample.  Results  indicated  moderate  stability  of  the 
empowerment  scale  oyer  time  (Spreitzer,  1995). 

A  second  order  confirmatory  factor  analysis  was  conducted  on  two 
samples  of  mid-leyel  employees  from  an  industrial  and  insurance 
organization  to  assess  the  conyergent  and  discriminant  yalidity  of  the 
empowerment  items  and  the  contribution  of  the  four  dimensions  to  the 
oyerall  construct  empowerment  (Spreitzer,  1995).  Results  indicated  that 
each  of  the  dimension  were  distinct  from  one  another.  Confirmatory 
factor  analysis  demonstrated  an  excellent  fit  for  the  industrial  sample,  and 
a  modest  fit  for  the  insurance  sample.  The  dimensions  also  demonstrated 
conyergent  yalidity  showing  relation  with  an  oyerall  construct  of 
psychological  empowerment.  Each  of  the  items  loaded  strongly  on  the 
appropriate  factor,  and  the  four  factors  were  significantly  correlated  with 
each  other  in  both  samples. 

While  results  demonstrated  that  the  four  dimensions  were  not  equiyalent, 
the  high  correlation  among  the  factors  (e.g.,  the  impact  scale  correlated  .43 
with  meaning  and  .63  with  self-determination)  leads  to  speculation 
whether  each  dimension  is  indeed  measuring  distinct  constructs. 

No  information  was  ayailable  to  guide  suggestions  on  the  application  of 
the  Empowering  Behayior  Questionnaire. 
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Job  Descriptiye  Index  (JDI) 

The  JDI  is  one  of  the  most  widely  used  measures  of  job 
satisfaction.  It  consists  of  fiye  scales:  work,  pay,  promotion, 
superyision,  and  co-workers.  For  each  scale,  respondents  respond 
to  a  list  of  short  phrases  indicating  if  the  phrase  applies  to  a 
particular  facet  of  his  or  her  job  (e.g.,  pay). 

Eyidence  in  the  literature  indicates  good  stability  coefficients  and 
internal  consistency  for  the  JDI  scales.  Schuler  (1979)  reported 
test-retest  coefficients  ranging  fi-om  .79  to  .82.  Roznowski  (1989) 
reported  alpha  coefficients  ranging  fi-om  .76  to  .88. 

Research  suggests  that  the  facets  of  job  satisfaction  measured  by 
the  JDI  appear  to  be  moderately  independent,  with  median 
interscale  correlations  ranging  between  .22  and  .43  (Cook, 
Hepworth,  Wall,  &  Warr,  1981).  Conyergent  and  discriminant 
yalidity  were  tested  by  comparing  the  JDI  facet  scales  to  related 
measures  such  as  the  Miimesota  Satisfaction  Questioimaire  (MSQ) 
and  the  Index  of  Organizational  Reactions  (lOR).  Dunham,  Smith, 
and  Blackburn  (1977)  compared  JDI  facet  scales  to  the  MSQ,  and 
reported  an  ayerage  conyergent  yalidity  coefficient  of  .47.  The 
discriminant  yalidity  results  reported  in  the  literature  are  somewhat 
less  promising  (e.g.,  Hartman,  Grigsby,  Crino,  &  Chokar’s  1986 
study  concluded  the  fiye  facets  of  the  JDI  met  only  two  of  the 
researchers’  three  criteria). 

The  scales  were  deyeloped  prior  to  the  rise  of  interest  in  intrinsic 
job  satisfaction  and  under  represent  such  facets  (e.g.,  helping 
others,  enjoying  social  aspects  of  work,  being  challenged, 
producing  quality  products). 

In  addition,  some  scales  are  shorter  than  others  in  terms  of  number 
of  items  per  scale.  Thus,  the  short  scales  are  more  likely  to  under 
represent  the  content  of  those  facets  of  job  satisfaction. 

The  JDI  can  be  used  to  assess  a  subordinate's  leyel  of 
satisfaction  with  the  superyisor.  Assessing  job  satisfaction  of 
subordinates  in  aggregate  can  help  to  eyaluate  organizational 
climate. 

Patricia  C.  Smith,  Lome  M.  Kendall,  or  Charles  L.  Hulin 
Bowling  Green  State  Uniyersity 
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Title:  Leader  Azimuth  Check  II  (Azimuth) 

Description:  The  Leader  Azimuth  Check  is  a  360-degree  multi-rater  assessment  that  is 

derived  from  the  Strategic  Leader  Development  Inventory  (SLDI)  which 
was  jointly  developed  by  the  Army  Research  Institute  (ARI),  the  Army 
War  College  (AWC),  and  the  Industrial  College  of  the  Armed  Forces 
(ICAF). 

There  are  two  current  versions  of  the  Leader  Azimuth  Check:  Azimuth 
Version  2.0  (Unit  and  Organizational  Use)  and  Azimuth  Version  2.1 
(CAS3  and  other  classroom  settings). 

Azimuth  (Version  2.0): 

The  96  items  that  made  up  the  first  Azimuth  (Version  1)  represented 
twelve  "elements"  of  leadership: 

Supervisory  Skills.  This  broad  element  of  leader  behavior  is  broken  out 
into  three  sub-elements:  interpersonal  focus,  team  focus  and  mission 
focus.  The  interpersonal  sub-element  examines  the  way  in  which 
supervisors  interact  with  subordinates.  The  team  focus  sub-element  looks 
at  leadership  in  developing,  motivating,  and  resourcing  of  teams.  Mission 
focus  examines  leaders  creating  an  environment  to  support  subordinates 
accomplishing  their  mission. 

Tactical  and  Technical  Competence.  Tactical  and  technical  competence  is 
also  broken  down  into  three  specific  sub-elements:  problem  solving  skills, 
knowledge,  and  planning/organizational  skills.  The  problem  solving  skills 
sub-element  looks  at  decision-makers'  ability  to  sort  out  the  important 
from  the  unimportant  details.  The  knowledge  sub-element  looks  at  the 
degree  to  which  leaders  eire  accomplished  professionals  in  their  technical 
or  tactical  area  of  expertise.  The  planning  and  organizational  skills  sub¬ 
element  is  concerned  with  leaders'  ability  to  meet  deadlines  and  maintain 
focus  on  primary  issues  even  when  other  things  compete  for  attention. 

Political  Skills.  This  element  is  concerned  with  sensitivity  to  political 
issues  and  interests  beyond  the  purely  military. 

Ethics.  This  element  is  concerned  with  personal  and  professional 
standards  of  conduct,  and  includes  the  ability  to  set  the  proper  example  of 
high  standards  of  interpersonal  responsibility  and  honesty. 

Communication/ Influence.  This  element  includes  the  establishment  and 
maintenance  of  both  forma!  and  informal  channels  of  communication  and 
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information  dissemination,  as  well  as  the  ability  to  provide  an  atmosphere 
conducive  to  openness  and  honesty. 

Social  Maturity.  This  element  is  concerned  with  a  leader’s  ability  to 
maintain  composure  under  conditions  which  may  be  personally 
threatening,  the  ability  to  admit  and  learn  from  mistakes,  and  the 
willingness  to  consider  the  opinions  of  others. 

Self-Centeredness.  This  is  a  negative  element  in  which  the  related 
behaviors  should  be  avoided.  This  element  is  centered  around  the  focus 
on  self  and  lack  of  concern  and  care  for  others. 

Compulsive  Behavior.  Like  the  element  above,  this  element  is  concerned 
with  behaviors  that  are  indicators  of  poor  leadership.  This  element 
examines  the  tendency  to  micromanage,  be  intolerant  of  ambiguity,  and  to 
desire  excessive  information  before  making  decisions. 

Feedback  from  the  Azimuth  is  structured  according  to  the  elements  and  is 
given  in  aggregate  and  not  item-by-item. 

Azimuth  ( Version  2. 1 1: 

The  72  items  in  the  current  Azimuth  (Version  2  and  Version  2.1)  were 
selected  to  represent  key  leadership  elements  in  the  new  Army  leadership 
doctrine  (FM  22-100).  The  items  were  derived  from  two  sources.  Some 
items  were  used  in  the  previous  version  of  Azimuth  and  were  retained 
because  feedback  from  the  several  thousand  officers  who  had  used  the 
prc\'ious  version,  and  statistical  evidence,  showed  that  most  respondents 
had  a  consistent  interpretation  of  these  items  (i.e.,  the  items  are 
statistically  “reliable”).  Additional  items  were  derived  directly  from  the 
element  definitions  and  descriptions  in  FM  22-100.  ARI  and  CAL 
personnel  collaborated  in  this  process,  and  a  draft  instrument  was  pilot 
tested  with  a  small  group  of  AY96-97  CGSOC  students.  One  goal  was  to 
keep  the  number  of  items  as  small  as  possible  to  keep  the  instrument  from 
becoming  unwieldy.  However,  five  or  six  short  items  cannot  possibly 
capture  all  of  the  nuances  of  a  complex  behavior  like  Communicating  or 
Decision  Making.  In  their  feedback  forms,  leaders  are  told  that  they  can 
assume  that  the  average  item  ratings,  i.e.,  the  element  scores,  are  a  good 
indicator  of  a  given  aspect  of  their  leadership  behavior;  however,  they 
should  not  assume  that  the  scores  are  an  absolute  assessment  of  their 
leadership. 

The  .Azimuth  items  are  divided  into  the  following  areas:  Communicating; 
Decision  Making;  Motivating;  Developing,  Building  (i.e.,  team  building); 
Learning  (i.e.,  supporting  a  learning  organization);  Planning  and 
Organizing;  Executing;  Assessing;  Respect;  Selfless  Service;  Integrity; 
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Emotional  Stability;  and  Other.  Two  general  items  are  analyzed  and 
reported  separately:  “This  person  is  a  good  leader”;  and  “This  person  is 
someone  I  would  follow  into  combat.”  Three  additional  items  solicit 
feedback  on  the  Azimuth  form  itself  and  on  the  data-collection  process. 

Feedback  is  presented  on  each  of  the  aspects  of  Leadership,  but  not  on  the 
individual  items  (with,  of  course,  the  exception  of  the  two  general  items). 
However,  individual  item  data  is  printed  out  for  use  by  a  mentor  or  advisor 
who  can  help  to  interpret  the  feedback  (see  Karrasch  and  Halpin,  1999). 

Azimuth  Version  2  was  designed  as  a  360-degree  instrument  for  use  in  a 
unit  or  organization.  Early  experience  with  a  pilot  implementation  within 
the  Combined  Arms  and  Services  Staff  School  (CASS)  led  to  a  decision  to 
develop  a  minor  variant.  Version  2.1,  for  use  within  an  academic  setting. 
Changes  include:  a)  provision  for  Self  and  Peer  input  only  (i.e.,  180- 
degree  rather  than  360-degree);  and  b)  rewording  of  items  to  reflect  the 
classroom  rather  than  an  organizational  setting.  The  Center  for  Army 
Leadership  (CAL)  has  also  developed  at  least  one  additional  variant  that 
eliminates  “negative”  items. 


Reliability  and 
Validity: 


Limitations: 


Application: 


Source: 


Periodic  checks  are  conducted  on  the  thousands  of  data  sets  that  have 
accumulated  through  the  use  of  Azimuth  in  operational  and  classroom 
settings  over  the  past  few  years.  The  Chronbach  alpha  for  all  elements  is 
typically  above  .80.  A  detailed  assessment  of  the  psychometrics  of  the 
instrument  has  been  accepted  as  a  dissertation  topic,  and  the  results  should 
be  available  in  early  2000. 

The  Azimuth  does  not  provide  an  absolute  assessment  of  leadership 
qualities  or  potential.  The  individual  items  in  the  instrument  provide  good 
coverage  of  important  leadership  behaviors,  however,  there  are  many 
other  aspects  of  leadership  which  are  not  addressed  by  this  instrument. 
Furthermore,  the  numeric  ratings  are  subjective  and  may  be  based  upon 
incomplete  information,  limited  opportunities  for  observation,  and 
potential  unconscious  biases. 

The  Azimuth  can  be  used  for  feedback  and  developmental  purposes  for 
the  leader.  The  Azimuth  should  be  completed  by  the  leader,  peers, 
subordinates,  and  the  supervisor. 

Dr.  Stanley  Halpin 

U.  S.  Army  Research  Institute 

Bldg  90  -  McClellan  Ave 

Fort  Leavenworth,  KS  66027-1360 

e-mail:  halpin@leav-emhl .army.mil 
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APPENDIX  B6 


Title: 

Description: 

Reliability: 

Validity: 


Limitations: 

Application: 

Source: 


Leader  Member  Exchange-7  (LMX-7) 

The  LMX-7  is  a  seyen-item  instrument  that  measures  how  leaders  use 
their  position  power  to  deyelop  different  exchange  relationships  with 
different  subordinates.  The  LMX-7  suggests  that  superyisors  employ  a 
social  exchange  framework  in  which  yarying  types  of  relationships  are 
established  with  subordinates. 

Scandura  and  Schriesheim  (1994)  reported  a  coefficient  alpha  reliability  of 
.86  for  the  LMX-7  measure. 

The  literature  on  leader-member  exchange  (e.g.,  Graen  &  Scandura,  1987) 
and  mentoring  (e.g.,  Scandura,  1992)  haye  established  clear  links  between 
the  deyelopmental  actiyities  of  superiors,  or  mentors,  and  the  career 
outcomes  of  subordinates,  or  proteges. 

Gerstner  and  Day  (1997)  conducted  a  meta-analysis  on  the  LMX-7 
measure  and  reported  significant  relationships  between  LMX  and  job 
performance,  satisfaction  with  superyision,  oyerall  satisfaction, 
commitment,  role  conflict,  role  clarity,  member  competence,  and  tumoyer 
intentions.  Thus,  the  meta-analysis  showed  that  the  LMX-7  measure  has 
soimd  psychometric  properties  and  that  the  LMX  is  congruent  with 
numerous  empirical  relationships  associated  with  transformational 
leadership. 

Dienesch  and  Liden  (1986)  suggested  that  LMX  studies  need  to  expand 
the  domain  of  yariables  examined  as  part  of  the  leader-subordinate 
interaction  process.  Augmenting  the  LMX  approach  with  aspects  of 
transformational  leadership,  such  as  mentoring,  may  be  yery  beneficial. 
Adding  mentoring  to  LMX  expands  the  boundaries  of  the  leader- 
subordinate  relationship  considerably. 

The  LMX-7  can  be  used  to  assess  the  leader’s  ability  to  commrmicate  with 
his  or  her  employees,  describes  how  leaders  use  their  position  power  to 
deyelop  different  exchange  relationships  with  different  subordinates.  For 
example,  when  high  leyels  of  leader-member  exchange  exist,  subordinates 
see  themselyes  as  haying  good  working  relationships  with  their 
superyisors  and  as  knowing  how  satisfied  their  supervisors  are  with  their 
performance.  Used  for  leader-subordinate  exchange  deyelopment 

George  B.  Graen 
P.O.Box  43131 
Lafayette,  LA  70504-3131 
e-mail:  gbgl010@usl.edu 
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APPENDIX  B7 


Title: 


Description: 


Reliability: 


Validity: 


Limitations: 

Application: 


Compass  -  the  Managerial  Practices  Suryey  (MPS) 

The  Managerial  Practice  Suryey  (MPS)  instrument  is  designed  to  proyide 
managers  with  information  about  their  current  behayiors  on  the  job,  to 
help  them  identify  their  strengths,  and  to  expand  their  repertoire  of 
effectiye  management  practices.  Based  on  a  15-year  research  program, 
MPS  measures  14  categories  of  management  and  leadership  behayiors. 
These  are:  informing,  clarifying,  monitoring,  planning,  problem  solying, 
consulting,  delegating,  inspiring,  recognizing,  rewarding,  supporting, 
mentoring,  networking,  and  team  building.  The  wording  of  the  items  on 
the  MPS  make  thern  suitable  for  use  by  peers  as  well  as  subordinates. 

As  cited  in  Clark  &  Clark  (1990),  Yukl,  Wall,  &  Lepsinger  reported  an 
internal  consistency  for  the  MPS  scales  that  ranged  from  .84  to  .91,  for  a 
sample  of  1,173  subjects.  Test-retest  reliability  was  tested  for  the  stability 
of  the  MPS  scales  and  results  ranged  from  .48  to  .94. 

Interrater  reliability  was  also  tested  for  agreement  between  subordinates 
with  respect  to  their  descriptions  of  the  manager’s  behayior.  The 
researchers  found  managers  differ  in  their  behayior  and  subordinate 
ratings  can  detect  this  difference.  The  F-Tests  were  significant  at  the  .01 
leyel  for  all  13  scales. 

Seyeral  studies  suggest  that  behayiors  in  the  MPS  are  releyant  for 
managerial  effectiyeness  (e.g..  Wall  and  Lepsinger,  1984, 1985  and  1986 
as  cited  in  Clark  &  Clark,  1990).  Content  yalidity  was  well  established 
with  correct  classification  ranging  from  72%  to  96%  (Taber  &  Falble, 
1984, 1985, 1986,  and  1988  as  cited  in  Clark  &  Clark,  1990).  Finally,  the 
criterion-related  yalidity  of  the  MPS  was  tested  in  seyeral  studies  (Yukl  & 
Kanuk,  1979;  Yukl  &  Van  Fleet,  1982;  Miles,  1985).  Across  all  studies, 
there  was  eyidence  for  the  releyance  of  most  criteria  for  managerial 
effectiyeness. 

Although  the  MPS’s  target  audiences  are  managers  at  all  leyels,  reliability 
and  yalidity  studies  were  conducted  among  mid-ley  el  managers  only. 

The  MPS  can  be  used  to  assess  subordinates’  perceptions  of  leader 
behayior.  Subordinates  complete  the  questionnaire  and  results  can  be  fed 
back  to  the  leaders  for  deyelopmental  purposes. 
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Source: 

Manus 

100  Prospect  Street,  South  Tower 
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Phone:  (203)  326-3890 

Fax:  (203)  326-3890 
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APPENDIXES 


Title: 


Description: 


Reliability: 


Validity: 


Multifactor  Leadership  Questionnaire  (MLQ) 

The  Multifactor  Leadership  Questionnaire  (MLQ)  is  an  80  item  measure 
of  transformational  and  transactional  leadership  skills.  The  MLQ 
measures  seyeral  dimensions  of  leadership  and  the  leader’s  perception  of 
his  or  her  effectiyeness.  The  MLQ  consists  of  seyeral  scales  measuring 
the  constructs  of  transformational  leadership,  transactional  leadership, 
organizational  outcome,  and  non-transactional  leadership.  The  fiye  scales 
that  measure  transformational  leadership  include  charisma,  inspiration, 
intellectual  stimulation,  and  indiyidualized  consideration.  The  three  scales 
measuring  transactional  leadership  include  contingent  reward,  actiye 
management  by  exception,  and  passiye  management  by  exception. 

Finally,  there  are  three  organizational-outcome  scales:  extra  effort, 
effectiyeness,  and  satisfaction  with  leader.  There  is  one  scale  for  non¬ 
transactional  leadership. 

The  leader  and  raters  across  three  organizational  leyels  (higher,  same  and 
lower)  complete  the  MLQ.  An  option  is  proyided  to  conceal  rater’s 
organizational  leyel. 

Alpha  reliability  coefficients  for  the  self-rating  form  range  from  .60  to  .92. 
Test-retest  reliabilities  of  the  self-rating  form  were  collected  oyer  a  six- 
month  period.  Reliabilities  for  the  factor  scales  range  fi^om  .44  to  .74 
(Bass  &  Ayolio,  1990). 

Criterion-related  yalidity  has  been  demonstrated  using  superyisors’ 
ratings,  promotion  reports,  and  financial  outcomes.  Both  transformational 
and  transactional  leadership  were  foimd  to  correlate  highly  with  these 
outcomes. 

Additionally,  eyidence  of  agreement  with  theory  is  demonstrated  by  the 
high  intercorrelations  among  the  four  transformational  factors  and  lower 
correlations  between  the  transformational  and  transactional  factors  (Bass 
&  Ayolio,  1990).  Lowe,  Kroeck,  &  Siyasubramaniam  (1996)  conducted  a 
meta-analytic  reyiew  of  39  published  and  unpublished  studies  that 
employed  the  MLQ  to  measure  transformational  leadership  and 
transformational  leadership  relationships,  including  charisma  and  leader 
effectiyeness.  The  reported  correlations  between  leader  charisma  and 
subordinate  ratings  of  effectiyeness  ranged  between  .36  and  .91,  with  a 
mean  corrected  correlation  of  .81 .  The  correlations  between  leader 
charisma  and  organizational  measures  of  effectiyeness  (either  direct 
measures  of  performance  or  performance  eyaluations)  ranged  between  .10 
and  .83,  with  a  mean  corrected  correlation  of  .35. 


B8-1 


T  .imitations:  Charismatic  and  inspirational  leadership  sub-scales  may  converge  to 

capture  a  global  conceptualization  of  charismatic  leadership.  Also,  it  has 
been  argued  that  the  assesses  a  single  higher  order  construct  of 
transformational  leadership  and  that  there  is  little  evidence  to  support  the 
contention  that  the  MLQ  measures  distinct  transformational  leader 
behaviors.  For  example,  correlation  coefficients  of  .79  and  .81  found 
among  the  transformation  factors  may  indicate  that  unique  factors  are  not 
being  measured. 

Application:  The  MLQ  can  be  used  to  measure  leadership  qualities  of  individuals  in  all 

types  of  organizations  and  all  organizational  levels.  Additionally,  the 
MLQ  can  be  used  to  assess  subordinates’  perceptions  of  leader 
effectiveness.  Depending  on  the  intended  use,  the  MLQ  should  be 
completed  by  the  leader  or  the  leader’s  subordinates. 

Source:  Mind  Garden,  Inc. 

P.O.  Box  60669 
Palo  Alto,  CA  94306 
Phone:  (415)  424-8493 
Fax:  (415)  424-0475 
e-mail:  mindgarden@msn.com. 
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Title: 

Description: 


Reliability: 

Validity: 

Limitations: 


Observer/Controller  Ratings  (0/C  Ratings) 

Observer/Controller  training  programs  provide  Observer/Controllers 
(0/C’s)  the  necessary  skills  to  assess  Battle  Staff  functioning  and 
effectiveness.  The  training  programs  were  designed  to  complement 
established  Combat  Training  Center  (CTC)  assessment  procedures  with 
the  additional  objective  of  assessing  individual  and  integrated  staff 
functional  skill  performance.  0/C’s  are  trained  to  identify  and  record 
critical  behaviors  related  to  battle  staff  effectiveness  and  report  critical 
incidents  for  effective  remediation. 

Development  of  the  0/C  training  program  is  based  on  earlier  theoretical 
work  on  the  Adaptive  Coping  Cycle  model  of  organizational  effectiveness 
(Olmstead,  1992)  which  consists  of  seven  processes:  sensing, 
communicating  information,  decision  making,  communicating 
implementation,  stabilizing,  coping  actions,  and  feedback.  For  purposes 
of  applying  the  conceptualization  of  the  Adaptive  Coping  Cycle  to  the 
Army,  the  terminology  was  modified.  Acquired  information  replaces 
sensing;  communicating  decisions  replaces  communicating 
implementation;  contingency  actions  replaces  stabilizing;  executing 
replaces  coping  actions;  and  feedback  and  follow-up  replaces  feedback. 
These  seven  processes  are  suggested  for  use  in  military  simulations  to 
determine  battle  staff  effectiveness. 

The  materials  used  to  train  O/Cs  include  an  Instructor  Guide,  a  Student 
Guide,  and  Course  Reference  Material.  The  Instructor  Guide  primarily 
provides  an  introduction  to  the  training  material,  training  objectives, 
lesson  plans  and  training  aids.  The  Student  Guide  primarily  provides  an 
overview  of  the  0/C  training  program,  training  course  objectives,  and 
references  relevant  to  functional  competence  of  organizations.  Finally,  the 
Course  Reference  Material  primarily  includes  a  guide  for  analyzing  and 
assessing  battle  staff  functions. 

Since  0/C  ratings  refer  to  the  generic  process  and  outcomes  of  0/C 
activities,  studies  discussing  an  overall  reliability  of  the  ratings  have  not 
been  conducted. 

Similarly,  studies  discussing  the  validity  of  0/C  ratings  have  not  been 
conducted. 

Some  of  the  0/C  training  program  instructions  have  not  been  translated 
into  Army  terminology.  Further,  detailed  vignettes  or  case  studies  from 
current  Army  Operations  should  be  included  in  each  instructional  miit  to 
better  relate  the  material  to  Army  doctrine  and  Battle  Staff  Operations. 


B9-1 


Application:  0/C  Ratings  has  been  used  for  assessing  soldiers’  performance  in  battle 

staff  simulations.  Additionally,  0/C  Ratings  were  used  for  training 
Observer  Controllers  in  the  ratings  of  battle  staff  effectiveness.  The 
template  may  be  adapted  for  assessing  performance  in  other  simulation 
environments. 

Source:  Scott  Graham 

Chief,  Infantry  Forces  Research  Unit 
U.S.  Army  Research  Institute 
5001  Eisenhower  Ave. 
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Tacit  Knowledge  for  Measure  of  Leadership  (TKML) 

The  Tacit  Knowledge  for  Military  Leadership  instrument  is  a  measure  that 
assesses  military  leader  tacit  knowledge.  Tacit  knowledge  can  be 
described  as  action-oriented,  practical  knowledge  that  can  be  acquired  on 
one’s  own  through  personal  experience  rather  than  instruction.  Tacit 
knowledge  in  military  leadership  consists  of  many  dimensions,  but  three 
haye  been  inyestigated  extensiyely.  The  three  dimensions  are  1) 
interpersonal  tacit  knowledge  which  requires  the  leader  to  manage 
him/herself  in  terms  of  organizing  one’s  own  self,  managing  time  and 
priorities,  seeking  challenges  and  control,  and  taking  responsibility;  2) 
interpersonal  tacit  knowledge  which  requires  the  leader  to  influence  and 
control  others,  support  and  cooperate  with  others,  and  learn  from  others; 
and  3)  organizational  tacit  knowledge  which  requires  the  leader  to  solye 
organizational  problems. 

The  TKML  consists  of  seyeral  work-related  situational  items  with  fiye  to 
twenty  responses.  Each  situation  poses  problems,  and  the  participant 
indicates  how  he  or  she  would  solye  it  by  rating  the  responses.  The 
difference  between  the  respondents’  set  of  ratings  from  the  standard  of 
experts’  ratings  for  all  of  the  situations  is  the  indiyidual’s  measure  of  his 
or  her  tacit  knowledge. 

The  TKML  instruments  haye  been  designed  for  use  on  battalion 
commanders,  company  commanders  and  platoon  leaders.  The  dimensions 
of  tacit  knowledge  for  the  battalion  commander  include:  communicating  a 
yision,  establishing  a  climate  for  deyelopment,  managing  the  leader  and 
the  subordinate,  proyiding  constancy,  and  using  influence  tactics.  The 
dimensions  of  tacit  knowledge  for  the  company  commander  include: 
caring  for  soldiers  through  task  completion,  prioritizing  and  solying 
problems,  proactiye  decision  making,  assessing  risk,  and  short  term 
decision  making.  The  dimensions  of  tacit  knowledge  for  the  platoon 
leader  are:  acquiring  confidence  in  interpersonal  skills,  defining  leadership 
style,  taking  a  stand,  and  taking  a  fostering  accoimtability. 

TKML  items  were  deyeloped  by  conducting  interyiews  with  actiye  duty 
Army  officers.  Tacit  knowledge  contained  in  the  interyiew  summaries 
was  identified  and  coded  by  two  researchers.  Interrater  reliability  was 
73%.  The  summaries  were  then  reyiewed  by  three  senior  military 
members  to  achieye  content  consensus  (Hedlund,  Horyath,  Forsythe, 
Snooks,  «&  Williams,  1998). 

Tacit  knowledge  has  been  foimd  to  increase  with  experience.  The  TKML 
was  found  to  discriminate  experienced  indiyiduals  from  those  who  are  not 
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experienced.  Testing  of  the  TKML  demonstrated  that  experienced  leaders 
displayed  higher  levels  of  tacit  knowledge  than  novice  leaders.  A 
discriminant  analysis  provided  support  that  experienced  and  novice 
leaders  responded  differently  to  the  TKML  items.  Canonical  correlation 
coefficients  were  R=.73,  p<.05;  R=.72,  p<.05;  R=.55,  p<.05  for  battalion 
commanders,  company  commanders,  and  platoon  leaders,  respectively 
(Hedlund,  et  al.,  1998). 

Criterion-related  validity  was  also  found  for  the  TKML.  The  instrument 
correlated  .3  to  .5  with  job  performance  measures. 

More  conclusive  evidence  of  the  generalizability  is  needed  for  the  TKML. 
Additionally,  further  research  on  the  construct  and  criterion-related 
validity  is  necessary  which  is  in  progress. 

The  TKML  may  be  useful  in  development  of  officer  training. 
Additionally,  the  TKML  may  be  used  as  part  of  a  selection  battery  for 
military  leaders.  Because  the  TKML  is  a  self-assessment  tool,  it  should 
only  be  completed  by  the  leader 
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Team  Effectiveness  Survey  (TES) 

The  Team  Effectiveness  Survey  (TES)  is  a  measure  designed  to  assess 
process  issues  associated  with  team  dynamics.  Specifically,  the  TES 
assesses  team  functioning  and  identifies  individuals  who  are  primarily 
responsible  for  the  team’s  style  of  functioning.  It  is  based  on  the 
assumption  that  team  effectiveness  will  improve  as  team  members 
understand  more  about  their  own  interactional  tendencies  and  discuss 
these  tendencies  with  team  members. 

The  TES  measures  individual  and  team  scores  of  exposure  (the  tendency 
to  engage  in  open  expressions  of  one’s  own  feeling  and  knowledge)  and 
feedback  (the  tendency  to  solicit  information  jfrom  others  about  their 
feelings  and  knowledge).  These  two  dimensions  are  proposed  to  influence 
the  effectiveness  of  communication  and  problem  solving.  Exposure  and 
feedback  scores  identify  four  types  of  individuals  and/or  teams: 

Type  A  -  low  feedback  and  low  exposure; 

Type  B  -  high  feedback  and  low  exposure; 

Type  C  -  low  feedback  and  high  exposme;  and 
Type  D  -  high  feedback  and  high  exposure.  These  profiles  serve  as 
immediate  feedback  to  confirm  or  deny  self-ratings  and  furnish  an 
overview  of  team  functioning. 

The  TES  also  measures  supportive  and  defensive  climates.  High 
defensive  climate  scores  and  low  supportive  climate  scores  indicate  that 
the  individual  has  a  constraining  effect  on  the  team  and  fosters  a  lack  of 
trust  among  team  members.  Conversely,  high  supportive  climate  scores 
and  low  defensive  scores  indicate  that  the  individual  works  well  with  the 
team  and  encourages  feelings  of  well-being.  Both  individual  and  team 
scores  are  calculated  with  the  TES.  Each  team  member  will  have  a  score 
on  the  four  dimensions  (exposure,  feedback,  defensive  climate,  and 
supportive  climate)  and  a  group  average  for  each  dimension. 

Inteijudge  reliability  has  been  established  as  .68  (Hall,  1996). 

No  construct  or  criterion-related  validity  studies  have  been  reported. 
Therefore,  the  TES  is  suitable  only  for  concept  training  and  as  a  stimulus 
to  team  discussion  (Hall,  1996). 

Concern  has  been  expressed  toward  the  construct  validity  of  the  TES.  For 
example,  it  is  proposed  that  exposure  and  feedback  are  not  independent 
constructs.  There  is  no  evidence  of  the  empirical  validity  of  the  scores  and 
inferences  of  team  or  individual  effectiveness.  Researchers  need  to 
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demonstrate  that  the  instrument  does  indeed  measure  team  effectiveness 
before  adopting  this  instrument. 

Application:  The  TES  can  be  used  for  team  building,  training  and  development,  or 

developmental  purposes.  It  is  designed  to  evaluate  individual  team 
members’  teamwork  practices.  This  information  can  be  used  to  help 
improve  team  functioning  and  communication.  The  TES  should  be 
completed  by  team  members  themselves. 

Somce:  Telemetries  International  Inc. 

755  Woodstead  Ct. 

Spring,  TX  77380 
Phone:  (281)  367-0060 
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Three-Component  Model  of  Organizational  Commitment 

Meyer  and  Allen’s  (1991)  Three-Component  Model  of  Organizational 
Commitment  contains  three  forms  of  commitment:  affectiye,  continuance, 
and  normatiye.  Affectiye  commitment  reflects  the  employee’s  emotional 
attachment  to,  identification  with,  and  inyolyement  in  the  organization. 
Employees  with  a  strong  affectiye  commitment  remain  with  an 
organization  because  the  want  to  do  so.  Continuance  commitment  refers 
to  an  awareness  of  the  costs  associated  with  leaying  an  organization. 
Employees  who  continue  employment  with  an  organization  based  on 
continuance  commitment  remain  because  they  need  to  do  so.  Finally, 
normatiye  commitment  reflects  a  feeling  of  obligation  to  continue 
employment.  Employees  with  a  strong  normatiye  commitment  feel  that 
they  ought  to  remain  with  the  organization  (Meyer  &  Allen,  1997). 
Common  to  these  three  approaches  is  the  yiew  that  commitment  is  a 
psychological  state  that  (a)  characterizes  an  employee’s  relationship  with 
the  organization  and  (b)  has  implications  for  the  decision  to  continue  or 
discontinue  employment  in  the  organization  (  Meyer,  Allen,  &  Smith, 
1993). 

The  Affectiye,  Continuance,  and  Normatiye  Commitment  Scales  (reyised) 
consists  of  6  items  each  (Meyer  et  al.,  1993).  Responses  to  each  item  are 
made  on  a  7-point  scale  with  anchors  labeled  (1)  strongly  disagree  and  (7) 
strongly  agree.  For  administration,  items  from  the  three  scales  are  mixed 
to  form  an  18-item  series 

Measures  of  the  three  components  of  organizational  commitment  were 
deyeloped  and  found  to  be  psychometrically  soimd  (Allen  &  Meyer, 

1990) .  Internal  consistency  for  the  three  scales  range  fi-om  a  low  of  20  for 
Normatiye  Commitment  Scale  to  a  high  of  40  for  the  Affectiye 
Commitment  Scale.  Median  reliabilities  for  the  Affectiye,  Continuance, 
and  Normatiye  Commitment  Scales  are  .85,  .79,  and  .73,  respectiyely. 
(Allen  &  Meyer,  1996;  Meyer  &  Allen,  1997).  Test-retest  reliability 
coefficients  range  from  .38  (Vandenberg  &  Self,  1993),  .60  (Meyer,  et  al., 
1993)  to  .94  (Blau,  Paul,  &  St.  John,  1993). 

Exploratory  (Allen  &  Meyer,  1990;  McGee  &  Ford,  1987;  Reilly  &  Orsak, 

1991)  and  confirmatory  (Dunham,  Grube,  &  Castaneda,  1994;  Hackett, 
Bycio,  &  Hausdorf,  1994;  Meyer.  Allen,  &  Gellatly,  1990;  Shore  & 
Tetrick,  1991;  Somers,  1993)  factor  analyses  support  that  affectiye, 
continuance,  and  normatiye  commitment  are  distinct  constructs.  Results 
of  factor  analyses  studies  also  proyide  eyidence  that  the  commitment 
constructs  are  distinguishable  from  related  constructs,  e.g.,  job  satisfaction 
(Shore  &  Tetrick,  1991),  career,  job,  and  work  yalues  (Blau  et.  al.,  1993), 
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career  eommitment  (Reilly  &  Orsak,  1991),  oecupational  commitment 
(Meyer  et  al.,  1993),  and  perceived  organizational  support  (Shore  & 
Tetrick,  1991). 

While  the  research  is  generally  supportive,  some  findings  suggest  the  need 
for  further  refinements  in  the  conceptualization  and  measurement  of 
commitment  (Meyer  &  Allen,  1997).  For  example,  the  continuance  scale 
may  comprise  two  related  dimensions:  laek  of  alternative  and  high 
personal  sacrifice  (Racket  et  al.,  1994;  McGee  &  Ford,  1987;  Meyer  et  al., 
1990;  Somers,  1993).  Other  studies  have  found  high  correlations  between 
affective  and  normative  scales  (Allen  &  Meyer,  1996;  Hackett  et  al.,  1994; 
Ko,  1997). 

Another  limitation  is  that  temporal  stability  appears  to  be  lower  when 
commitment  is  measured  early  in  employees’  eareers  (e.g.  less  then  one 
month  as  studied  by  Vandenberg  &  Self,  1993)  though  it  does  begin  to 
stabilize  fairly  quickly  (e.g.,  after  one  month  as  studied  by  Meyer  et  al., 
1993).  Therefore,  these  scales  may  not  be  appropriate  for  use  with  new 
employees. 


U.S.  Army  Research  Institute  (ARI)  has  employed  a  15-item  measure 
based  on  the  Meyer  and  Allen  Three-Component  Scales  (Oliver,  Tiggle,  & 
Hayes,  1996).  The  Meyer  and  Allen  instrument  was  modified  by 
substituting  “the  military”  for  “my  organization”  and  deleting  items  which 
did  not  apply  to  the  military.  These  scales  can  also  be  modified  by  using  a 
5-point  Likert  scale.  Each  scale  can  also  be  used  independently.  For 
example,  Oliver  et  al.  (1996)  employed  only  two  seales  (affeetive  and 
continuance  eommitment)  using  a  5 -point  scale  with  results  mirroring 
similar  Three-Component  Model  studies  (e.g.,  McGee  &  Ford,  1987). 

While  the  Three-Component  Scales  were  developed  as  an  independent 
questionnaire,  their  brevity  affords  the  opportunity  to  integrate  the  items 
into  or  with  other  measures  of  interest.  Also,  the  measures  can  be  usefully 
employed  as  both  an  diagnostic  or  research  tool. 
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APPENDIX  B 13 


Title: 

Description: 


Reliability: 

Validity: 

Limitations: 

Application: 


U.S.  Army  Automated  Command  Climate  Suryey 

The  U.S.  Army  Automated  Command  Climate  Suryey  is  a  fully 
automated,  self-contained,  suryey  program.  It  is  designed  to  assist  the 
company  (or  equiyalent-sized  unit)  commander  in  assessing  and 
deyeloping  action  plans  for  sustaining  and  improying  his  or  her  unit 
command  climate. 

The  Command  Climate  Survey  consists  of  24  basic  questions  and  two 
comment  questions,  which  address  20  command  climate  areas.  In 
addition,  the  unit  commander  can  add  up  to  ten  optional  unit-specific 
questions.  The  20  climate  areas  include:  officer  leadership,  NCO 
leadership,  immediate  supervisor,  leader  accessibility,  leader  concern  for 
families,  leader  concern  for  single  soldiers,  unit  cohesion,  counseling, 
training,  racist  materials,  sexually  offensive  materials,  stress,  training 
schedule,  sponsorship,  respect,  unit  readiness,  morale,  sexual  harassment, 
discrimination,  and  reporting  harassment/discrimination  incidents. 
Additional  questions  on  gender  and  race  are  asked.  Written  comments  are 
requested  regarding  unit  strengths  and  areas  most  needing  improvement. 

The  survey  results  may  provide  indicators  of  strengths  and  issues  in  a  unit. 
In  addition,  the  program  automatically  encrypts  the  data  to  protect 
soldiers’  anonymity.  The  survey  results  help  the  commander  determine 
his  or  her  unit  climate  and  assist  in  development  and  implementation  of 
actions  for  improvement. 

Company  commanders  are  encouraged  to  perform  a  Command  Climate 
Survey  within  90  days  of  assuming  conunand  to  help  them  assess  a  variety 
of  issues,  including  unit  readiness,  racial  and  sexual  harassment  climate, 
and  morale.  Army  officials  indicate  this  survey  will  likely  become  a 
requirement. 

No  reliability  studies  have  yet  been  performed. 

No  reliability  studies  have  yet  been  performed. 

No  evidence  of  validation  or  reliability  has  been  reported. 

The  Command  Climate  Survey  can  be  used  to  identify  problems  in  a 
commander’s  unit  and  help  to  develop  solutions.  The  survey  is  designed 
as  a  self-contained,  stand-alone  tool  for  the  commander.  The  commander 
can  administer  the  survey,  score  or  tally  the  results,  and  conduct  the 
analysis  to  assess  his  or  her  unit’s  current  climate. 
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Source:  Morris  Peterson 

ATTN:  TAPC-ARI-PS 

U.  S.  Army  Research  Institute  for  the  Behavioral  Sciences 

5001  Eisenhower  Avenue 

Alexandria,  VA  22333-5600 

Phone:  (703)  617-7803 

DSN:  767-7803 

e-mail:  peterson@ari.army.mil 
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APPENDIX  B14 


Title; 

Description: 


Reliability: 

Validity: 

Limitation: 

Application: 

Sources: 


Walter  Reed  Army  Institute  of  Research  (WRAIR)  Leadership 
Assessment  Tool 

This  measure  uses  human  dimensions  to  assess  and  monitor  how  changes 
in  work  enyironments  impact  soldiers  and  leaders.  This  measure  has 
proyided  a  method  of  (1)  quantifying  and  recording  how  the  work 
enyironment  of  soldiers  and  leaders  change  as  a  function  of  external 
factors  and  (2)  estimating  the  organizational  impact  of  these  changes. 

The  measure  has  the  following  scales,  each  testing  a  different  human 
dimension:  task  significance,  support  for  task  significance,  support  for 
unit’s  mission  information,  dissemination  unit  leadership/yertical 
cohesion,  peer  relations/horizontal  cohesion,  job  satisfaction,  and 
collectiye  efficacy. 

Bliese  and  Halyerson  (1996, 1998),  Marlowe  (1986),  and  Cammann, 
Fichman,  Jenkins  &  Klesh,  (1983)  haye  tested  the  reliability  of  the  scales. 
All  researchers  foxmd  moderately  high  Cronbach  Alpha  coefficients  (.81  to 
.92).  Howeyer,  the  role  clarity  and  work  oyerload  scales  showed 
moderately  low  reliability. 

No  yalidity  studies  were  completed  on  this  measure. 

The  most  significant  limitation  of  the  measure  is  the  lack  of  data 
supporting  the  yalidity  of  the  measure.  Further,  while  the  entire  package 
of  scales  is  conyenient,  more  current  measurement  scales  are  ayailable  for 
the  indiyidual  dimensions  (e.g.,  commitment). 

The  WRAIR  assessment  tool  can  be  can  be  tested  on  rmit  soldiers  to 
identify  what  external  factors  influence  soldiers  and  their  working 
enyironments.  Results  fi'om  the  assessment  can  be  used  to  eyaluate 
organizational  climate. 

MAJ  Paul  D.  Bliese 

Department  of  Operational  Stress  Research 
Walter  Reed  Army  Institute  of  Research 
Phone:  (301)  295-7856. 
e-mail:  bliese@wrair-emhl  .army.mil 
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APPENDIX  B15 


Title: 


Description: 


Reliability: 


Validity: 


Limitations: 


360-Degree  Assessments 

The  term  360  degree  feedback  refers  to  the  practice  of  gathering  and 
processing  multirater  assessments  on  indiyiduals  and  then  feeding  back 
the  results.  Typically,  a  360-degree  instrument  is  a  questionnaire  with 
about  100  items  to  rate.  Approximately  10  people  (raters)  complete  it:  the 
person  being  assessed  (self-rating),  his  or  her  supervisor,  seyeral  peers, 
and  subordinates.  It  is  recommended  that  360-degree  feedback  reports  be 
confidential  (between  superyisor  and  respondent)  and  that  an  outside 
consultant  deliyer  assessment  feedback  and  coaching. 

In  the  majority  of  organizations,  360-degree  feedback  is  used  for 
deyelopmental  purposes.  Ratings  are  collected  anonymously  and  proyided 
to  managers  in  the  aggregate.  Usually,  only  the  managers  being  rated  see 
the  feedback  report.  The  ratings  are  not  included  in  the  managers'  formal 
performance  appraisal. 

A  study  inyestigated  within-source  interrater  reliability  of  superyisor,  peer, 
and  subordinate  feedback  ratings  made  for  managerial  deyelopment. 

Raters  proyided  360-degree  feedback  ratings  on  a  sample  of  153 
managers.  Using  generalizability  theory,  results  indicated  little  within- 
source  agreement  exists  (Harris  &  Schaubroeck,  1988). 

Harris  and  Schaubroeck  (1988)  conducted  a  meta-analysis  and  found 
relatiyely  high  correlations  between  peer  and  supervisor  ratings,  but  only  a 
moderate  correlation  between  self-supervisor  and  self-peer  ratings.  They 
also  found  that,  while  rating  format  and  rating  scale  had  little  impact,  job 
type  appeared  to  moderate  self-peer  and  self-supervisor  ratings.  The 
analysis  showed  a  higher  convergence  between  observer  (peer  and 
supervisor)  ratings  than  self  and  observer  ratings.  Self-peer  and  self¬ 
supervisor  ratings  are  particularly  low  for  managerial/professional  staff. 

Disagreement  among  rating  sources  is  both  common  and  expected.  In  a 
hierarchical  organization,  the  360-degree  approach  violates  the  hierarchy 
of  organizational  practices.  Research  suggests  that  upward  feedback  can 
substantially  affect  the  acceptance  of  feedback  for  managers.  Bemardin, 
Dahmus,  &  Redmon  (1993)  demonstrated  that  supervisors  are  generally 
accepting  of  upward  feedback,  but  they  are  less  supportive  of  this 
feedback  if  it  only  comes  from  subordinates. 

Also,  360-degree  feedback  assessments  may  have  problems  with 
anonymity.  Often,  raters  will  fear  being  identified  in  rating  their 
supervisor  despite  attempts  at  confidentiality. 
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Application:  A  360-degree  assessment  can  be  used  as  a  performance  appraisal  tool  or 

for  developmental  purposes.  Implicit  in  the  measure,  a  360-degree 
assessment  should  be  completed  by  the  supervisor,  self,  peers,  and 
subordinates. 

Source:  The  Center  for  Creative  Leadership 

One  Leadership  Place 
Greensboro,  NC  27410 
Phone:  (336)  288-7210 
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Example  Assessment  Templates:  Observer/Controller  Rating  Forms 


FORSCOM  LEADERSHIP  DEVELOPMENT  PROGRAM 
LEADERSHIP  ACTIONS  FEEDBACK  GUIDE 
(For  U»e  By  NTC  Observer/Controllers) 

ARMY  DEFINITION  OF 
LEADERSHIP 

(FM  ZaoOQ,  DRAH  S«pitmbfi  IMA) 

*Lt»Oership  to  the  process  ot  Influenclns  people  -  by 
provMtnp  purpose,  direcllon,  end  mollvstron  •  while 
opereling  lo  eccomplish  the  mission  and  while 
Improving  the  orgsnlutlon." 

BE  KNOW  DO 
ARMY  LEADERSHIP  DOCTRINE 
I  L  eaarn  ot  Character  and  Competence 


WIlM*  SMVk* 
Honoi 


Act  to  Achieve  Bacellenee  by  Providing 
Purpose,  Direction,  and  Motivation 


ACTIONS 

-DO'’ 

INFLUENCING 

OPERATING 

IMPROVING 

Cemmuniceling 

Planning 

Developing 

Dtcialen^Ung 

Executing 

Building 

MeOveting 

Ataeaaing 

Learning 

LEADERSHIP  ACTIONS 


Side  2 

Piscusetonf^? 

- II 


•  towM  VVeming  Order,  reisiued  as  . .  I  1 

•  Restoled  missiort:  issued  timely,  ergsnized  OPORO. . .  1  j 

•  Communiesled  >  desrty  6  eon^ely  •  Mem  artd 

ooncepi  to  subordinBies  (indudes  Task  ft  Puipose)....~...~~..~~...|  ] 

•  UtHixed  biief*bad«.  rehearsals,  teedbeck  lo  ensure 

thorough  undemanding  d  inslruclions. _ _  ( ] 

•  Fostered  •  spM  of  Candor  d  Bhe  k  1 1 

gccnten-W«Mnt 

•  Gathered  Mmeslirnales  rtecessary  to  support  dediion.»..^.~.~.,~~~{  ] 

•  Syrrlhesieed  ft  analysed  Inlo;  focused  on  critical  isaues........^..~~..-  j  | 

•  Made  decisioo  basM  on  sound  )udgntenl,  logical  anslysis........>.......i  ] 

•  Anrtounced  lintely  decision  desriy  ft  eondsely  j 

fttotfvatinp 

•  Meinlalned  dimale  which  erwoursged  high  periormBnce......^.....~.~.| ) 

•  Demonsireled  concern  for  safety  ft  protection  ol  subordinates.....^^  j  j 

•  Provided  encouragement,  supp^  direction,  ft  eorrectionM..M.~.....~..  I  j 

•  Kepi  unit  intormed:  provided  timety  leedbacK . . . . . I  1 

Ptanninp 

•  Specified  additional  kilo  requirements . . . ] 

•  Planned,  akocsled.  ft  managed  j  j 

•  Evaluated  kilo  (considered  source,  accuracy,  relevance}.................  (  j 

•  Coordkiaied  emptoymenl  ot  ALL  assets  (orgarric/nomorganic)......  j  j 

•  Sal  dear  priorities;  esiabtisheo  contiDls/lriggers........~..............~..  I  1 

txeeatinp 

•  Followed  plan;  adjusted  whenhrrhere  necessary,  clearly 
announced  adiustments.......................................................  |  1 

•  Provided  command/leadei  presence  ('right  time,  right  piece')........  (  j 

•  Adjusled  rapidly  lo  obstacles,  ditlicullies,  unloreseen  events........  j  j 

•  Guide<yiBught/coached  subordinales  during  execution...... . .  |  j 

Aaaeaalnp 

•  Aware  el  curreni  status  el  key  kerns;  sought  additional  required  No.|  ] 

•  Assessed  current  status  agakisl  desired  end  stale.....................  (  j 

•  Reported  status  ft  assessmeni  to  HHO  ki  timely  m8nner...............|  j 


ggjre/gp/nfl 

•  Communicaled  expectattons:  established  high  standards..............  | ) 

•  Established  coacMngTleachingrmenloring  as  an  ongoing  process....  (  j 

•  Knew  strengths  ft  weakness  ol  aubordinates...... . .  j  j 

•  Sel  the  example............................................................  I  j 

Buildinp 

•  Developed  ft  reMorced  aense  ol  teamwork........................^.  (  ] 

•  Esiablished  conditions  lor  demonstration  el  kiifeiive  by  aubordinales  (  j 
Laaminp 

•  SougN  ways  lo  improve;  translated  previous  experience  Me  acfton.^  1 

•  Fostered  e  teaming  dimale;  estabtished  junctional  leedback  aystem..j  j 
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Assessment  Measurement  Feedback 

Please  complete  the  following  information  regarding  your  use  of  assessment  measures  and  mail,  fax,  or  e-mail  your 
response  to  ARI  as  detailed  on  the  next  page.  Your  responses  and  comments  will  help  guide  future  “tool  kit” 
recommendations  which  will  help  enhance  the  effectiveness  of  future  leadership  research  for  the  Army. 

From:  _  Date: 


Research  Title:  _ _ 

Primary  Investigator:  _  Other  Investigators: 

Date  Study  Initiated/Completed:  _ -  , _ 

Piupose  of  Study:  _ _ 


Please  place  a  check  by  all  the  following  research  topics  that  your  research  addressed.  If  your  research  addressed 
linkages  between  these  topic  areas,  please  draw  lines  between  fee  appropriate  topics. 

_ Leadership  competencies,  _ Leadership  performance 

skills  and  temperament  requirements 

_ Leadership  selection  and  assessment  _ Individual,  unit,  and 

organizational  effectiveness 

_  Leadership  training  and  development  _  Other:  _ _ 


Leadership  assessment  measure  used:  _ _ _ 

Using  fee  framework  below,  enter  your  assessment  measure  into  fee  appropriate  cell. 


MEASURES  OF  LEADER  EFFECTIVENESS 


Process  Measures 

Outcome  Measures 

Organizational  Level: 

Lower 

Middle 

Lower 

Middle 

Level 

of 

Analysis: 

Individual 

Dyad 

Team 

Unit 

Overall,  how  useful  was  this  tool  kit  report  to  you? 

1  2  3  4  5 

Not  At  All  Useful  Not  Very  Useful  Somewhat  Useful  Useful  Very  Useful 


Continued  on  next  page 
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Written  Comments 


Please  comment  on  the  effectiveness  as  well  as  lessons  learned.  As  you  realize,  your  written  comments  provide 
valuable  insight  on  your  experience  with  the  measure.  Thank  you  for  providing  this  quality  information. 


Thank  you  for  providing  feedback.  Yoii  responses  and  comments  are  a  valued  part  of  continual  improvement  in  the 
leadership  research  community.  Please  mail,  fax,  or  e-mail  yoiu  response  to  ARI  as  detailed  below. 

Army  Research  Institute 

Organizational  &  Personnel  Resources  Research  Unit 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5000 
Phone:  (703)  617-8866  or  DSN  767  -  8866 
Fax:  (703)617  8578 
e-mail:  gade@ari.army.mil 


APPENDIX  E 


Leadership  Assessment  Measures  Workshop 
Participant  List 


Aude,  Steve^ 

Klein,  Katherine* 

Avolio,  Bruce* 

LeBoeuf,  Joseph* 

Ayman,  Roya* 

Lewis,  Phillip* 

Bliese,  PauP 

McGee,  Michael* 

Dardis,  Greg^ 

Michael,  Rex* 

Day,  David* 

Pstoka,  Joseph* 

Drillings,  Michael^ 

Ruskin,  Robert* 

Gade,  Paul* 

Rumsey,  Michael* 

Hedlund,  Jennifer* 

Shaler,  Michael* 

Hunt,  Jerry* 

Simutis,  Zita* 

Jacobs,  Owen* 
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APPENDIX  F 


Background  and  Pre-Workshop  Assignment  Information  Mailed  to  Workshop  Participants 

Background  Information 

This  project  involves  identifying  a  set  of  existing  leader  effectiveness  measures,  e.g.,  the 
MSQ,  or  measurement  approaches,  e.g.,  a  360°  approach,  that  meet  professional  standards  and 
then  promoting  their  use  in  future  Army  leadership  research  and  operational  settings.  The 
measures  that  we  identify  with  your  help  ,  which  survive  scrutiny,  by  our  panel  of  experts  will  be 
systematically  organized  and  then  made  available  to  investigators  and  operators  who  wish  to  use 
them.  These  measures  would  be  used  whenever  possible  in  ARI  sponsored  research.  This 
means  that  a  measure  identified  by  the  project  would  be  included  if  it  fits  the  investigator's 
conceptual  framework  for  effectiveness  and/or  the  operational  context  of  interest,  e.g..  Army 
War  College.  Alternatively,  when  there  is  a  DESIRE  to  innovate  and  create  or  try  out  a  NEW 
measure  of  effectiveness  in  a  study,  the  investigator  would  be  encouraged  to  also  use  as  a 
benchmark  the  most  comparable  scale/approach  found  in  our  "tool  kit."  Finally,  when  there  is  a 
NEED  to  irmovate  because  there  is  no  suitable  measure,  the  investigator  would  be  encouraged  to 
place  the  new  scale  into  the  nomological  network  of  constructs  by  using  several  referent 
measures  identified  by  this  project. 

The  focus  of  this  phase  of  our  project  is  to  organize  the  domain  of  leadership 
effectiveness  and  to  identify  suitable  existing  measures  or  measurement  approaches  or  templates. 
In  this  regard,  you  will  see  that  we  are  asking  you  to  use  a  framework  that  we  have  created  for 
this  purpose.  Our  fi-amework  has  partitioned  the  effectiveness,  "criterion  space,"  in  a  particular 
manner  and  along  certain  dimensions.  The  first  dimension  emphasizes  the  fact  that  leader 
effectiveness  can  be  manifested  and  measured  in  terms  of  PROCESSES  and/or  OUTCOMES. 
Similarly,  either  measures  of  process  or  outcomes  may  be  used  to  capture  effectiveness  at 
different  organizational,  hierarchical  levels.  Here  we  nominate  three.  Finally  we  assert  that 
measures  may  be  differentially  suitable  for  and  focused  on  a  particular  unit  of  analysis.  For 
purposes  of  the  pre-work  and  the  workshop  itself,  we  are  asking  you  to  entertain  four  such  units  - 
individual,  dyad,  group  or  team,  and  business  or  command.  The  combination  of  factors  listed 
above  partitions  the  criterion  into  24  feasible  cells. 

It  is  important  to  note  that  a  specific  measure  or  type  of  measure  may  indeed  be  suitable 
for  more  than  one  location  in  this  "criterion  space."  In  fact,  we  are  open  to  this  possibility. 
However,  we  do  not  want  to  assume  this  at  the  outset.  It  is  also  likely  that  no  good  measure  or 
approach  exists  for  some  of  the  cells  in  our  fi'amework.  But  this  too  has  to  be  established. 
Finally,  we  are  sensitive  to  the  possibility  that  still  other  views  can  be  offered  regarding  the 
manifestation  of  leader  effectiveness.  There  may  well  be  additional  factors  that  must  be 
considered  before  we  can  readily  identify  or  nominate  a  potentially  useful  measure.  The  utility 
or  appropriateness  of  a  measure  might  depend  on  such  things  as  the  presumed  purpose"  or  "use" 
of  the  measure  (e.g.,  to  measure  effectiveness  in  training  or  for  administrative  action),  the  setting 
in  which  the  measure  is  to  be  taken  (e.g..  Army  garrison  context,  war  games/simulations),  and 
even  whether  we  are  interested  in  assessing  the  effectiveness  of  civilian  vs.  military  leaders. 

This  last  point  withstanding,  we  ask  that  you  try  to  work  within  the  fi’amework  that  we  have 
provided  in  this  packet  and  leave  these  latter  factors  for  discussion  at  the  workshop. 
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Pre-Workshop  Assignment 


We  would  like  you  to  carefully  consider  the  framework  below  and  use  it  to  organize  your 
thinking  about  the  leadership  effectiveness  measures  with  which  you  are  f^iliar.  Relying  on 
professional  standards,  you  are  then  to  nominate  specific,  even  proprietary  measures  for  as  many 
of  the  cells  in  the  framework  as  you  wish.  As  noted  above,  it  is  possible  for  a  given  measure  to 
be  nominated  for  more  than  one  location.  On  the  other  hand,  it  is  also  possible,  even  desirable  to 
nominate  more  than  one  measure  for  a  given  cell.  Of  course,  despite  your  efforts,  some  cells  may 
remain  empty. 

It  is  important  to  point  out  that  the  measures  you  should  consider  may  derive  from  basic 
or  applied  research.  They  may  have  been  used  in  either  or  both  civilian  and  military  contexts. 

The  key  point  here  is  that  you  feel  that  they  have  the  strong  potential  to  be  useful  as  a  way  to 
index  or  define  leader  effectiveness. 


When  a  plausible  measure  or  approach  has  been  identified,  its  name  should  be  entered  on 
the  framework  schematic  in  the  appropriate  place.  When  possible,  please  also  cite  relevant 
references.  Finally,  please  bring  a  copy  of  each  measure  you  nominate  to  the  workshop. 


Measures  of  Leader  Effectiveness 

Process  Measures 

Outcome  Measures 

Organizational 

Level: 

Lower 

Middle 

Upper 

Lower 

Middle 

Upper 

I 

H 

Individual 

> 

Dyad 

Team 

Unit 
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Army  Leader  Assessment  Workshop  Agenda 
Friday,  November  13, 1998 


Start-End  Time 

Event 

8:30-9:00 

Buffet  Breakfast 

9:00-10:15: 


10:15-11:30 


11:30-12:15 


12:15-1:15 


1:15-3:00 


3:00-5:00 


5:00-5:30 


Welcome  and  Introductory  Comments 


Facilitated  discussion:  current  leader  assessment  practices  in  the  Arm 


Facilitated  discussion:  criteria  for  effective  assessment  tools 


Lunch 


Presentation,  nomination,  and  discussion  of  specimen  measures 


Small  group  discussion 


Recap  and  preparation  for  second  da 


Saturday,  November  14, 1998 


Start-End  Time 

Event 

8:30-9:00 

Breakfast  Buffet 

9:00-10:30 

Small  group  discussion:  arriving  at  consensus 

10:30-12:15 

Presentation  and  facilitated  discussion  of  conclusions  from  small  group 
discussions 

12:15-1:00 

Wrap  up  and  discussion  of  future  direction 

1:00-2:00 

Lunch 
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APPENDIX  H 


Group  1  -  Individual 
Greg  Dardis 
Mike  Drillings 
Phil  Lewis 
Joe  Psotka 
Ken  Wexley 


Leadership  Assessment  Measures  Workshop 
Small  Group  Sessions* 

Group  2  -  Dvad 
David  Day 
Paul  Gade 
Owen  Jacobs 
Herman  Keizer,  Jr. 
Rex  Michael 


Gary  Yukl 


Group  3  -  Team 
Steve  Aude 
Paul  Bliese 
Jennifer  Hedlund 
Bob  Kilcullen 
Katherine  Klein 


Group  4  -  Unit 
Roya  Ayman 
Jerry  Himt 
Joe  LeBoeuf 
Mike  McGee 
Mike  Rumsey 
Scott  Tannenbaum 


*  Four  workshop  members  were  imable  to  participate  in  the  small  group  sessions 
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Please  substitute  the  attached  pages  for  the  previously-submitted 
pages  B5-1  through  B5-4  of  ARI  Research  Note  99-35 
(ADA368448). 


